Elamur Research⌕
☰
home / papers / 2018 / Neural Machine Translation Decoding with Terminology Constraints

Neural Machine Translation Decoding with Terminology Constraints

Summary

Businesses that translate manuals or web content often need specific terms, such as brand names, rendered exactly as their terminology prescribes. Standard NMT gives no such guarantee. Earlier constrained decoders could also put a term in the wrong position or translate its source words a second time. The authors combine finite-state acceptors (FSAs), which encode every permutation of the required phrases and their synonym alternatives, with multi-stack beam search in the style of Anderson et al. Their main addition uses the decoder's attention weights as soft alignments. A constraint is inserted only when the model attends most strongly to the source span aligned with it. Once the constraint is complete, attention to that span is zeroed out, so it is not translated again. They evaluated the method on WMT17 English↔German and English↔Chinese, using ensembled Blocks models combined with SMT lattices (L NMT). With dictionary constraints, BLEU rose by as much as 3 points while overall quality held steady. Attention-guided decoding also produced fewer repetitions and ran considerably faster.

Key points

  • Attention-guided placement keeps only one constraint permutation, which cuts decoding complexity from O(tk2^c) to O(tkc).
  • Two kinds of constraints were tested:
    • Gold: up to two reference tokens, or one phrase of up to five subwords, that the baseline output lacked.
    • Dictionary: term pairs drawn from the references and matched against a pruned phrase table.
  • On the English-German dev set, decoding with attentions lowered the count of repeated character 7-grams from 610 to 471.
  • Speed was measured on newstest2017 with an English-German production system (beam 4):
    • Attention-based decoding (v2*) was 3.5× faster than the Anderson-style decoder at c=2 and more than 5× faster at c=4.
    • Its BLEU was slightly lower (27.8 vs 28.2 at c=2).
  • Relaxations recover quality at some cost in speed, reaching up to 28.7 BLEU against the 26.7 L NMT baseline. They include secondary-attention thresholding and allowing 1–2 extra tokens around a constraint.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

79 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 76 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-2081

@inproceedings{hasler2018neural,
  title = {{Neural Machine Translation Decoding with Terminology Constraints}},
  author = {Hasler, Eva and Gispert, Adrià de and Iglesias, Gonzalo and Byrne, Bill},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-2081},
  url = {https://doi.org/10.18653/v1/n18-2081}
}

References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chris Hokamp and Qun Liu. 2017. Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 1535–1546. https://doi. org/10.18653/v1/P17-1141… find our card
  2. Tamer Alkhouli and Hermann Ney. 2017. Biasing Attention-Based Recurrent Neural Networks Using External Alignment Information. In Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers. Association for Computational Linguistics, pages 108–117. http://www. statmt.org/wmt… find
  3. Gonzalo Iglesias, William Tambellini, Adrià de Gispert, Eva Hasler, and Bill Byrne. 2018. Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry… find
  4. Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2017. Guided Open Vocabulary Image Captioning with Constrained Beam Search. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 936– 945. https://… find our card
  5. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. Transactions of the As… find
  6. Dzmitry Bahdanau, KyungHyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of ICLR 2015. https://arxiv.org/ pdf/1409.0473.pdf. ↗ our card
  7. Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 Conference on Machine Translat… find our card
  8. Minh-Thang Luong and Christopher D Manning. 2015. Stanford Neural Machine Translation Systems for Spoken Language Domains. In Proceedings of the 12th International Workshop on Spoken Language Translation. pages 76– 79. https://nlp.stanford.edu/pubs/ luong-manning-iwslt15.pdf. ↗
  9. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving Neural Machine Translation Models with Monolingual Data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. pages 86–96. https://aclweb.org/anthology/P16-1009. ↗
  10. Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, and Frédéric Blain. 2017. Guiding Neural Machine Translation Decoding with External Knowledge. In Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers. Association for Computational Ling… find
  11. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2017. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 1715–1725. https://aclweb. org/anthology/P… find
  12. Josep Crego, Jungi Kim, Guillaume Klein, Anabel Rebollo, Kathy Yang, Jean Senellart, Egor Akhanov, Patrice Brunelle, Aurélien Coquard, Yongchao Deng, Satoshi Enoue, Chiyo Geiss, Joshua Johanson, Ardas Khalsa, Raoum Khiari, Byeongil Ko, Catherine Kobus, Jean Lorieux, Leidiana Martins, Dang-Chuan Ngu… find
  13. Felix Stahlberg, Adrià de Gispert, Eva Hasler, and Bill Byrne. 2017. Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Shor… find
  14. Bart van Merriënboer, Dzmitry Bahdanau, Vincent Dumoulin, Dmitriy Serdyuk, David Warde-Farley, Jan Chorowski, and Yoshua Bengio. 2015. Blocks and Fuel: Frameworks for deep learning. In Proceedings of ICLR 2015. Arxiv preprint. https://arxiv.org/abs/1506.00619. ↗
  15. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, and Yoshua Bengio. 2017. On integrating a language model into neural machine translation. Computer Speech & Language 45:137– 148. https://doi.org/10.1016/j.csl. 2017.01.014. — [doi:10.1016/j.csl](https://doi.org/10.1016/j.csl) ↗
  16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hasler-2018-neural-machine-translation-decoding-terminology/