Elamur Research
☰
home / papers / 2018 / Marian: Fast Neural Machine Translation in C++

Marian: Fast Neural Machine Translation in C++

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1804.00344

Summary

Most neural machine translation toolkits rely on Python and general-purpose deep-learning libraries, which can make them slower and harder to optimize. Marian addresses this with an NMT framework written entirely in C++11 and deliberately shipped without Python bindings. It has its own back end that performs reverse-mode automatic differentiation over dynamic computation graphs, similar in spirit to DyNet. The back end is tuned for translation workloads, for example through fused RNN cells, attention, and layer normalization. A modular encoder/decoder interface lets researchers combine architectures, such as an RNN encoder with a Transformer decoder, by implementing only a single inference step. The authors rebuilt the winning UEdin WMT17 English–German pipeline in Marian and matched its quality. Swapping in a Transformer improved results further, and training was much faster than with Nematus. Marian also produced strong systems for automatic post-editing and grammatical error correction. The result is a fast, open-source (MIT) toolkit that remains practical for research and has already been deployed in production at WIPO.

Key points

  • Marian depends only on Boost plus CUDA or a BLAS library. It originated as a C++ reimplementation of Nematus and can still load its common models, and Intel contributed a CPU back end.
  • On the WMT17 en-de replication, a single Transformer reached 35.6 BLEU on newstest-2016 and 28.8 on newstest-2017, compared with 33.9 and 27.5 for the UEdin single model. A 4-model ensemble with R2L reranking reached 36.8 and 29.5, a gain of 1.2 BLEU on newstest-2017 over the original system.
  • Nematus trains a shallow model at roughly 2,800 source tokens/s. Marian trains identical models about 4× faster on one GPU and about 30× faster on 8 GPUs, with near-linear multi-GPU scaling.
  • Back-translating 10M sentences with a shallow model took about 4 hours on 8 GPUs, at roughly 15,850 tokens/s with beam size 5.
  • In automatic post-editing at WMT-2017, Marian-based systems ranked second on automatic metrics and first in human evaluation. In grammatical error correction, they beat the best previous neural systems by more than 8% M² on CoNLL-2014 and more than 4.5% on JFLEG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics: Machine translation

Cited by

422 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 412 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1804.00344 · arXiv 1804.00344

@misc{junczysdowmunt2018marian,
  title = {{Marian: Fast Neural Machine Translation in C++}},
  author = {Junczys-Dowmunt, Marcin and Grundkiewicz, Roman and Dwojak, Tomasz and Hoang, Hieu and Heafield, Kenneth and Neckermann, Tom and Seide, Frank and Germann, Ulrich and Aji, Alham Fikri and Bogoychev, Nikolay and Martins, André F. T. and Birch, Alexandra},
  year = {2018},
  doi = {10.48550/arxiv.1804.00344},
  eprint = {1804.00344},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1804.00344}
}

References

17 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Roee Aharoni and Yoav Goldberg. 2016. Sequence to sequence transduction with hard monotonic attention. arXiv preprint arXiv:1611.01487. find
  2. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450. find our card
  3. Ondrej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno-Yepes, Philipp Koehn, and Julia Kreutzer, editors. 2017a. Proc. of the 2nd Conference on Machine Translation, WMT 2017. Association for Computational Linguistics. find
  4. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017b. Findings of the 2017 Conference on Machine Transla… find our card
  5. Yarin Gal and Zoubin Ghahramani. 2016. A theoretically grounded application of dropout in recurrent neural networks. In Advances in neural information processing systems, pages 1019–1027. find our card
  6. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2017a. The AMU-UEdin submission to the WMT 2017 shared task on automatic post-editing. In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers, pages 639–646, Copenhagen. Association for Computational Linguistics. find
  7. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2017b. An exploration of neural sequence-to-sequence architectures for automatic post-editing. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 120–129. Asian Federation of Natural… find
  8. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In Proceedings of NAACL-HLT 2018, New Orleans, USA. Association for Computational Linguistics. Accepted for publication. find
  9. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, pages 67–72. find our card
  10. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In ACL.… find
  11. Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Pau… find our card
  12. Ofir Press and Lior Wolf. 2017. Using the output embedding to improve language models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 157–163. find our card
  13. Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, and Philip Williams. 2017a. The University of Edinburgh’s neural MT systems for WMT17. In (Bojar et al., 2017a), pages 389–399. find
  14. Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nadejde. 2017b. Nematus: a toolkit for neural machine translation. In Proceedings of the Software Demonstrations o… find our card
  15. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li… find
  16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… find
  17. Barret Zoph and Kevin Knight. 2016. Multi-source neural translation. CoRR, abs/1601.00710. find