Elamur Research
☰
home / papers / 2015 / Minimum Risk Training for Neural Machine Translation

Minimum Risk Training for Neural Machine Translation

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1512.02433

Summary

Neural machine translation (NMT) is usually trained with maximum likelihood estimation (MLE). MLE optimizes word-level cross-entropy on gold prefixes, not the sentence-level metrics such as BLEU that measure translation quality, and it never shows the model its own outputs. The authors adapt minimum risk training (MRT), long used in statistical MT, to end-to-end NMT. The objective becomes the expected loss over the model's distribution of translations, for example negative smoothed sentence-level BLEU. Enumerating every candidate is intractable, so they draw a small sample of translations from the model, add the reference, and renormalize probabilities over this subset using a sharpness hyper-parameter α. The loss does not need to be differentiable, and the approach works with any network architecture. On Chinese-English, English-French and English-German, MRT applied to the attention-based RNNsearch system consistently outperformed MLE, with the biggest gains on Chinese-English. The work offers a simple, architecture-agnostic way to tune generation models directly toward evaluation metrics. It was later reused for headline generation with ROUGE.

Key points

  • Approximating the risk: Sampling about 100 candidates per sentence (k = 100) proved sufficient, and larger samples gave no significant benefit. Duplicate samples are discarded, and GPU parallelism speeds up sampling.
  • Data:
    • Chinese-English used 2.56M sentence pairs, with NIST 2006 for validation and NIST 2002–2005 and 2008 for testing.
    • English-French used the standard 12M-pair WMT 2014 subset, with news-test 2014 as the test set.
  • English-French (case-sensitive BLEU): MLE scored 29.88 and MRT scored 31.30. Adding Luong et al.'s PosUnk rare-word handling raised MRT to 34.23, above earlier systems listed in the paper.
  • English-German: MLE scored 16.45 and MRT scored 18.02, or 20.45 with PosUnk. This is close to a much deeper LSTM model (20.90).
  • Why Chinese-English gained most: The authors suggest two reasons. Its test sets have four references per sentence, and the two languages are structurally far apart. They also contrast MRT with MIXER (Ranzato et al., 2015), which uses a single sample for its REINFORCE reward.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics: Machine translation

Cited by

197 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 182 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1512.02433 · arXiv 1512.02433

@misc{shen2015minimum,
  title = {{Minimum Risk Training for Neural Machine Translation}},
  author = {Shen, Shiqi and Cheng, Yong and He, Zhongjun and He, Wei and Wu, Hua and Sun, Maosong and Liu, Yang},
  year = {2015},
  doi = {10.48550/arxiv.1512.02433},
  eprint = {1512.02433},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1512.02433}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ayana, Shiqi Shen, Zhiyuan Liu, and Maosong Sun. 2016. Neural headline generation with minimum risk training. arXiv:1604.01904. — [arXiv:1604.01904](https://arxiv.org/abs/1604.01904) find
  2. Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR. find our card
  3. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguisitics. find
  4. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL. find our card
  5. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of EMNLP. find
  6. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of HLT. find
  7. Jianfeng Gao, Xiaodong He, Wen tao Yih, and Li Deng. 2014. Learning continuous phrase representations for translation modeling. In Proceedings of ACL. find
  8. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. arXiv:1503.03535. find our card
  9. Xiaodong He and Li Deng. 2012. Maximum expected bleu training of phrase and lexicon translation models. In Proceedings of ACL. find
  10. Sebastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In Proceedings of ACL. find our card
  11. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proceedings of EMNLP. find
  12. Philipp Koehn and Hieu Hoang. 2007. Factored translation models. In Proceedings of EMNLP. find
  13. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of HLT-NAACL. find
  14. Alon Lavie and Michael Denkowski. 2009. The mereor metric for automatic evaluation of machine translation. Machine Translation. find
  15. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Proceedings of ACL. find
  16. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015a. Effective approaches to attention-based neural machine translation. In Proceedings of EMNLP. find
  17. Minh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. 2015b. Addressing the rare word problem in neural machine translation. In Proceedings of ACL. find our card
  18. Franz J. Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of ACL. find
  19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of ACL. find our card
  20. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv:1511.06732v1. find our card
  21. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Improving neural machine translation models with monolingual data. arXiv:1511.06709. find
  22. David A. Smith and Jason Eisner. 2006. Minimum risk annealing for training log-linear models. In Proceedings of ACL. find
  23. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of AMTA. find our card
  24. Andreas Stolcke. 2002. Srilm - am extensible language modeling toolkit. In Proceedings of ICSLP. find
  25. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS. find
  26. Ronald J. Willams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning. find