Elamur Research⌕
☰
home / papers / 2020 / Boosting Neural Machine Translation with Similar Translations

Boosting Neural Machine Translation with Similar Translations

Paper · 2020 · Jitao Xu, Josep Crego and Jean Senellart

Summary

Human translators speed up their work by consulting fuzzy matches from translation memories. Neural MT, however, has no built-in way to use such matches beyond training or fine-tuning on the memory. This paper builds on the "neural fuzzy repair" approach of Bulté and Tezcan (2019), in which each NMT input is extended with the target side of the best-scoring translation-memory entry. The authors compare three retrieval criteria:

  • edit-distance fuzzy matching,
  • longest n-gram overlap,
  • cosine similarity of sent2vec sentence embeddings, searched with faiss.

A tighter integration uses fast_align alignments and the longest common subsequence to mark which retrieved target words correspond to the input. These marks are passed to the network as an additional factor stream. On several data sets and domains, the augmented systems consistently outperform baselines and earlier work. Lexical matches mostly give the model words to copy, while embedding matches appear to enrich context. The two kinds of match, together with fine-tuning, add up. Inference overhead is negligible, and a C++ fuzzy-matching toolkit is released openly.

Key points

  • Fuzzy score is defined as one minus edit distance divided by the longer sentence length. An example where a helpful match scores only 0.583 motivates the extra n-gram matching measure.
  • In the factor-based variant, each input token is tagged S (source word), T (related target word) or R (unrelated target word).
  • Adding FM/EM matches on top of fine-tuned models yields roughly +6 BLEU on average over fine-tuned baselines and about +2.5 over FM/EM on generic models.
  • Usage analysis shows the "copy" versus "context" effect:
    • output usage of T words rises by more than 10% compared with base models;
    • R words rise less;
    • embedding-match (E) words rise only modestly.
  • Released resource: a suffix-array-based fuzzy-match library (github.com/systran/FuzzyMatch). Other tools used are sent2vec, faiss and fast_align.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France104 works · 60,877 citations🔬Laboratoire d'Informatique pour la Mécanique et les Sciences de l'IngénieurOrsay · France21 works · 7,271 citations🎓Université Paris-SaclayGif-sur-Yvette · France34 works · 5,733 citations🏢SystranParis · France3 works · 151 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 958 works

Cite

Cite as Jitao Xu, Josep Crego and Jean Senellart. 2020. Boosting Neural Machine Translation with Similar Translations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.144

@inproceedings{xu2020boosting,
  title = {{Boosting Neural Machine Translation with Similar Translations}},
  author = {Jitao Xu, Josep Crego and Jean Senellart},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.144},
  url = {https://doi.org/10.18653/v1/2020.acl-main.144}
}

References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Bloodgood and Benjamin Strauss. 2014. Translation memory retrieval methods. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 202–210. find
  2. Bram Bulté and Arda Tezcan. 2019. Neural fuzzy repair: Integrating fuzzy matches into neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1800–1809, Florence, Italy. Association for Computational Linguistics.
  3. Bram Bulté, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. M3tra: integrating tm and mt for professional translators. pages 69–78. EAMT. find
  4. Chenhui Chu and Rui Wang. 2018. A survey of domain adaptation for neural machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1304–1319, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  5. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, pages 67–72, Vancouver, Canada. Association for Computational Linguistics.
  6. Catherine Kobus, Josep Crego, and Jean Senellart. 2017. Domain control for neural machine translation. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 372–378, Varna, Bulgaria. IN-COMA Ltd. find
  7. Philipp Koehn and Jean Senellart. 2010. Convergence of Translation Memory and Statistical Machine Translation. In Proceedings of AMTA Workshop on MT Research and the Translation Industry, pages 21–31, Denver.
  8. Xiaoqing Li, Jiajun Zhang, and Chengqing Zong. 2018. One sentence one model for neural machine translation. In Proceedings of the 11th Language Resources and Evaluation Conference, Miyazaki, Japan. European Language Resource Association. find
  9. Minh-Thang Luong and Christopher D. Manning. 2015. Stanford neural machine translation systems for spoken language domain. In International Workshop on Spoken Language Translation, Da Nang, Vietnam. find
  10. Udi Manber and Gene Myers. 1993. Suffix arrays: a new method for on-line string searches. siam Journal on Computing, 22(5):935–948. find
  11. Annual Conference of the European Association for Machine Translation, pages 153–160, Antalya, Turkey. find
  12. Jan Niehues, Eunah Cho, Thanh-Le Ha, and Alex Waibel. 2016. Pre-translation for neural machine translation. CoRR, abs/1610.05243. find
  13. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces…
  14. John E Ortega, Felipe Sánchez-Martınez, and Mikel L Forcada. 2016. Fuzzy-match repair using black-box machine translation systems: what can be expected. In Proceedings of AMTA, volume 1, pages 27–39. find
  15. Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. 2018. Unsupervised learning of sentence embeddings using compositional n-gram features. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lo…
  16. Mike Paterson and Vlado Dančı́k. 1994. Longest common subsequences. In International Symposium on Mathematical Foundations of Computer Science, pages 127–142. Springer. find
  17. Mirko Plitt and François Masselot. 2010. A productivity test of statistical machine translation post-editing in a typical localisation context. The Prague bulletin of mathematical linguistics, 93:7–16. find
  18. Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, and Armand Joulin. 2019. Ccmatrix: Mining billions of high-quality parallel sentences on the web. arXiv preprint arXiv:1911.04944.
  19. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, San Dieg…
  20. Kun Wang, Chengqing Zong, and Keh-Yih Su. 2013. Integrating translation memory into phrase-based machine translation during decoding. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11–21, Sofia, Bulgaria. Association for Comp… find
  21. Jitao Xu, Josep Crego, and Jean Senellart. 2019. Lexical micro-adaptation for neural machine translation. In International Workshop on Spoken Language Translation, Honk Kong, China. find
  22. Masaru Yamada. 2011. The effect of translation memory databases on productivity. Translation research projects, 3:63–73. find
  23. Anna Zaretskaya, Gloria Corpas Pastor, and Miriam Seghiri. 2015. Integration of machine translation in cat tools: State of the art, evaluation and user attitudes. Skase Journal of Translation and Interpretation, 8(1):76–89. find
  24. Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig, and Satoshi Nakamura. 2018. Guiding neural machine translation with retrieved translation pieces. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog…
  25. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational L…
  26. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA). find
  27. Tom Vanallemeersch and Vincent Vandeghinste. 2015. Assessing linguistically aware fuzzy matching in translation memories. In Proceedings of the 18th find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2020-boosting-neural-machine-translation-similar/