Elamur Research⌕
☰
home / papers / 2016 / Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

Summary

The paper tackles automatic post-editing (APE): repairing the output of an unseen machine translation system (English→German, IT domain) in the WMT 2016 shared task. The AMU team applied attention-based neural encoder-decoder models to this problem. The official post-editing data was far too small, so they created millions of synthetic triplets. They filtered German Common Crawl text by cross-entropy and passed it through German→English→German round-trip translation with phrase-based systems. They then kept the triplets whose TER statistics most closely resembled the real APE data. They trained two kinds of NMT models: monolingual ones (MT output → post-edit) and bilingual ones (English source → post-edit). Their ensembles were merged in a tuned log-linear model with a simple Post-Editing Penalty that discourages words absent from the input. This setup captures both the MT output and the source sentence and keeps corrections conservative. The final system ranked first in the shared task by a wide margin. It showed early on that neural models, given enough synthetic data, can beat phrase-based APE approaches.

Key points

  • Data: 12,000 official training triplets and 1,000 development triplets, plus about 10 million artificial triplets from round-trip translation. These were filtered by TER similarity into round-trip.n10 (4,335,715 triplets) and round-trip.n1 (531,839 triplets).
  • Preprocessing used the Moses tokenizer and truecaser and BPE subword units with 40,000 units per language. German MT output and post-edits shared one vocabulary.
  • Models: four mt→pe and four src→pe checkpoints were combined log-linearly in AmuNMT. The Post-Editing Penalty adds a −1 cost for each output word not seen in the input.
  • On the development set, the full system improved over the raw MT baseline by −3.7% TER and +6.0% BLEU. It also gained −1.8% TER and +2.1% BLEU over the monolingual-only ensemble.
  • On the test set, the system scored 21.52 TER and 67.65 BLEU, against 24.76 TER and 62.11 BLEU for uncorrected MT (−3.2% TER, +5.5% BLEU). The best other team, FBK, reached 23.92 TER.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

70 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 65 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w16-2378

@inproceedings{junczysdowmunt2016log,
  title = {{Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing}},
  author = {Junczys-Dowmunt, Marcin and Grundkiewicz, Roman},
  booktitle = {Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers},
  year = {2016},
  doi = {10.18653/v1/w16-2378},
  url = {https://doi.org/10.18653/v1/w16-2378}
}

References

24 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648, Atlanta, Georgia. Association for Computational Linguistics. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations, San Diego, CA. find our card
  3. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, (2):23–38. find
  4. Hanna Béchara, Yanjun Ma, and Josef van Genabith. 2011. Statistical post-editing for a statistical MT system. In Proceedings of the 13th Machine Translation Summit, pages 308–315, Xiamen, China. find
  5. Ulrich Germann. 2015. Sampling phrase tables for the Moses statistical machine translation system. Prague Bulletin of Mathematical Linguistics, (1):39–50. find
  6. Hanna Béchara, Raphaël Rubino, Yifan He, Yanjun Ma, and Josef van Genabith. 2012. An evaluation of statistical post-editing systems applied to RBMT and SMT systems. In Proceedings of COL-ING 2012, pages 215–230, Mumbai, India. find
  7. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 690–696, Sofia, Bulgaria. find our card
  8. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 Workshop on Statistical Machine Translation. In Proc… find our card
  9. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for… find
  10. Rajen Chatterjee, Marco Turchi, and Matteo Negri. 2015a. The FBK participation in the WMT15 automatic post-editing shared task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 210–215, Lisbon, Portugal. Association for Computational Linguistics. find
  11. Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 220– 224, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  12. Rajen Chatterjee, Marion Weller, Matteo Negri, and Marco Turchi. 2015b. Exploring the planet of the APEs: a comparative study of state-of-the-art methods for MT automatic post-editing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Internationa… find
  13. Santanu Pal, Mihaela Vela, Sudip Kumar Naskar, and Josef van Genabith. 2015. USAAR-SAPE: An English–Spanish statistical automatic post-editing system. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 216–221, Lisbon, Portugal. Association for Computational Linguistics. find
  14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Stroudsburg, PA, USA. Association for Computational Lingu… find our card
  15. Colin Cherry and George Foster. 2012. Batch tuning strategies for statistical machine translation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 428–436, Stroudsburg, PA, USA. Association for C… find
  16. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015a. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709. find
  17. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua… find
  18. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015b. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. find
  19. Michel Simard, Cyril Goutte, and Pierre Isabelle. 2007. Statistical phrase-based post-editing. In Proceedings of the Conference of the North American Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the Chapter… find
  20. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas, pages 223–231, Cambridge, Massachusetts. find our card
  21. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: 28th Annual Conference on Neural Information Processing Systems 2014, pages 3104– 3112, Montreal, Canada. find
  22. Guillaume Wisniewski, Nicolas Pécheux, and François Yvon. 2015. Why predicting post-edition is so hard? failure analysis of LIMSI submission to the APE shared task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 222– 227, Lisbon, Portugal. Association for Computatio… find
  23. Matthew D. Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. find our card
  24. Barret Zoph and Kevin Knight. source neural translation. arXiv:1601.00710. 2016. Multi-arXiv preprint — [arXiv:1601.00710](https://arxiv.org/abs/1601.00710) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/dowmunt-2016-log-linear-combinations-monolingual-bilingual/