Elamur Research
☰
home / papers / 2022 / Survey of Low-Resource Machine Translation

Survey of Low-Resource Machine Translation

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2022.cl-3.6.pdf

Summary

Most of the world's roughly 7,000 languages, and almost every language pair, lack enough translated text to train neural MT systems. Such systems typically rely on tens or hundreds of millions of sentence pairs. Haddow and co-authors map out how researchers handle this scarcity, focusing on NMT.

They group the work into several strands:

  • finding or crawling more parallel and monolingual data,
  • exploiting monolingual text,
  • transferring knowledge from other language pairs through multilingual models,
  • adding other linguistic resources,
  • training and modeling more effectively with little data.

The authors also discuss evaluation. They review techniques used in 18 low-resource shared tasks held between 2018 and 2021, identifying what top systems commonly used.

The results are uneven. Large multilingual models with plenty of synthetic data and a moderate parallel corpus can reach good quality. Many other pairs still get very weak output, and languages beyond the 100–200 studied so far are largely unserved.

The survey is a practical entry point for newcomers. It names future priorities: working with language communities, understanding massive multilingual models, using external knowledge, and making systems robust against hallucinations.

Key points

  • Opus counts show how unequal resources are: about 280M parallel sentences for English–French, about 0.7M for English–Myanmar, and only about 35k for English–Fon. Fon's only Opus corpus, JW300, has since been withdrawn over copyright.
  • Opus covers over 500 languages and variants. Wikipedia has text in more than 300 languages, and the WMT news monolingual corpus covers 59. Web-derived resources include Oscar, CC100, mc4, Paracrawl and Samanantar (Indian languages).
  • Shared-task outcomes vary widely. German–Upper Sorbian reached BLEU near 60 at WMT21, while AmericasNLP scores were almost all under 10. The best system in the 2021 Large-scale Multilingual "full" track averaged about 16 BLEU.
  • The authors stress separating simulated low-resource setups from genuinely low-resource languages. Genuinely low-resource languages also face noisy, out-of-domain data (often religious or IT localization texts) and weak basic tools such as language identification.
  • Community efforts such as Masakhane, GhanaNLP and AI4Bharat are highlighted, along with workshops including AfricaNLP, AmericasNLP, DravidianLangTech and BSNLP.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

113 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 110 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00446

@article{haddow2022survey,
  title = {{Survey of Low-Resource Machine Translation}},
  author = {Haddow, Barry and Bawden, Rachel and Barone, Antonio Valerio Miceli and Helcl, Jindřich and Birch, Alexandra},
  journal = {Computational Linguistics},
  year = {2022},
  doi = {10.1162/coli\_a\_00446},
  url = {https://doi.org/10.1162/coli_a_00446}
}

References

128 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adelani, David Ifeoluwa, Dana Ruiter, Jesujoba O. Alabi, Damilola Adebonojo, Adesina Ayeni, Mofe Adeyemi, Ayodele Awokoya, and Cristina España-Bonet. 2021. MENYO-20k: A multi-domain English-Yorùbá corpus for machine translation and domain adaptation. CoRR, abs/2103.08647. Agić, Željko and Ivan… find our card
  2. Aharoni, Roee, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874–388… find our card
  3. Arivazhagan, Naveen, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. Massively multilingual neural machine translation in the wild: Findings and challenges. In Proceedi… find our card
  4. Artetxe, Mikel, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In Proceedings of the 6th International Conference on Learning Representations. https://doi.org /10.18653/v1/D18-1399 Artetxe, Mikel and Holger Schwenk. 2019a. Margin-based parallel corpus m… ↗ our card
  5. Ataman, Duygu, Wilker Aziz, and Alexandra Birch. 2019. A latent morphology model for open-vocabulary neural machine translation. In Proceedings of the 7th International Conference on Learning Representations. find
  6. Ataman, Duygu, Matteo Negri, Marco Turchi, and Marcello Federico. 2017. Linguistically motivated vocabulary reduction for neural machine translation from Turkish to English. The Prague Bulletin of Mathematical Linguistics, 108:331–342. https://doi.org/10.1515 /pralin-2017-0031 Babych, Bogdan, Olga K… find
  7. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations. Bañón, Marta, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Go… find our card
  8. Conference on Machine Translation (WMT19). In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 1–61. https://doi.org/10.18653/v1/W19-5301 Barrault, Loı̈c, Magdalena Biesialska, Ondrej Bojar, Marta R. Costa-jussà, Christian Federmann, Yvette Gr… ↗ our card
  9. Bawden, Rachel, Nikolay Bogoychev, Ulrich Germann, Roman Grundkiewicz, Faheem Kirefu, Antonio Valerio Miceli Barone, and Alexandra Birch. 2019. The University of Edinburgh’s submissions to the WMT19 news translation task. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shar… find
  10. Bertoldi, Nicola and Marcello Federico. 2009. Domain adaptation for statistical machine translation with monolingual resources. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 182–189. https://doi.org/10.3115 /1626431.1626468 Bhattacharyya, Sumanta, Amirmohammad Roosh… find
  11. Bojanowski, Piotr, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. https://doi.org/10.1162/tacl\_a\_00051 Bojar, Ondřej and Aleš Tamchyna. 2011. Improving translation mod… ↗
  12. Bowman, Samuel R., Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 10–21. https://doi.org /10.18653/v1/K16-1002 Briakou, Eleft… ↗ our card
  13. Brown, Tom, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Ch… find
  14. Buck, Christian, Kenneth Heafield, and Bas van Ooyen. 2014. N-gram counts and language models from the Common Crawl. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3579–3584. find
  15. Budiwati, Sari Dewi, Al Hafiz Akbar Maulana Siagian, Tirana Noor Fatyanosa, and Masayoshi Aritsugi. 2019. DBMS-KU interpolation for WMT19 news translation task. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 141–146. https://doi.org/10.186… find
  16. Chen, Peng Jen, Ann Lee, Changhan Wang, Naman Goyal, Angela Fan, Mary Williamson, and Jiatao Gu. 2020. Facebook AI’s WMT20 news translation task submission. In Proceedings of the Fifth Conference on Machine Translation, pages 113–125. find
  17. Cheng, Yong, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Semi-supervised learning for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1965–1974. https://doi .org/10.18653/v… find
  18. Christodouloupoulos, Christos and Mark Steedman. 2015. A massively parallel corpus: The Bible in 100 languages. Language Resources and Evaluation, 49(2):375–395. https://doi.org/10.1007 /s10579-014-9287-y Chronopoulou, Alexandra, Dario Stojanovski, and Alexander Fraser. 2021. Improving the lexical a… find
  19. Clark, Stephen and James R. Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493–552. https://doi.org/10.1162/coli.2007 .33.4.493 Conneau, Alexis, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisc… ↗ our card
  20. Cover, Thomas M. and Joy A. Thomas. 2006. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). find
  21. Wiley-Interscience, USA. Currey, Anna, Antonio Valerio Miceli Barone, and Kenneth Heafield. 2017. Copied monolingual data improves low-resource neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 148–156. https://doi.org/10.18653/v1/W17 -4715 Dabre, Raj,… ↗
  22. Dabre, Raj, Tetsuji Nakagawa, and Hideto Kazawa. 2017. An empirical study of language relatedness for transfer learning in neural machine translation. In Proceedings of the 31st Pacific Asia Conference on Language, Information and Computation, pages 282–286. find
  23. Dandapat, Sandipan and Christian Federmann. 2018. Iterative data augmentation for neural machine translation: A low resource case study for English-Telugu. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation, pages 287–292. Daumé, Hal and Daniel Marcu. 2… find
  24. Denkowski, Michael and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380. https://doi.org/10.3115 /v1/W14-3348 Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kri… find our card
  25. Dhar, Prajit, Arianna Bisazza, and Gertjan van Noord. 2020. Linguistically motivated subwords for English-Tamil translation: University of Groningen’s submission to WMT-2020. In Proceedings of the Fifth Conference on Machine Translation, pages 126–133. find
  26. Di Gangi, Mattia Antonino and Marcello Federico. 2017. Monolingual embeddings for low resourced neural machine translation. In Proceedings of the 14th International Workshop on Spoken Language Translation, pages 97–104. find
  27. Ding, Shuoyang, Adithya Renduchintala, and Kevin Duh. 2019. A call for prudent choice of subword merge operations in neural machine translation. In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, pages 204–213. find
  28. Dinu, Georgiana, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan. 2019. Training neural machine translation to apply terminology constraints. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3063–3068. https://doi.org/10 .18653/v1/P19-1294 Do… find our card
  29. Edman, Lukas, Antonio Toral, and Gertjan van Noord. 2020. Low-resource unsupervised NMT: Diagnosing the problem and providing a linguistically motivated solution. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 81–90. find
  30. Edunov, Sergey, Myle Ott, Michael Auli, and David Grangier. 2018a. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500. https://doi.org/10 .18653/v1/D18-1045 Edunov, Sergey, Myle Ott, Michael Auli, David Gr… find
  31. Eikema, Bryan and Wilker Aziz. 2019. Auto-encoding variational neural machine translation. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 124–141. https://doi.org/10.18653/v1/W19-4315 ↗
  32. Eikema, Bryan and Wilker Aziz. 2020. Is MAP decoding all you need? The inadequacy of the mode in neural machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4506–4520. https://doi .org/10.18653/v1/2020.coling-main .398 Emezue, Chris Chinenye an… find
  33. Fadaee, Marzieh, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low-resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567–573. https://doi.org /10.18653/v1/P17-2090 Fan, Ang… ↗ our card
  34. Feng, Fangxiaoyu, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic BERT sentence embedding. CoRR, abs/2007.01852. find our card
  35. Feng, Yang, Shiyue Zhang, Andi Zhang, Dong Wang, and Andrew Abel. 2017. Memory-augmented neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1390–1399. https://doi.org/10.18653/v1/D17-1146 ↗
  36. Finn, Chelsea, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1126–1135. find
  37. Firat, Orhan, Kyunghyun Cho, and Yoshua Bengio. 2016. Multi-way, multilingual neural machine translation with a shared attention mechanism. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 866–875… find our card
  38. Freitag, Markus and Orhan Firat. 2020. Complete multilingual neural machine translation. In Proceedings of the Fifth Conference on Machine Translation, pages 550–560. find
  39. Freitag, Markus, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021a. Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460–1474. https://doi.org/10.1162… find our card
  40. Garcia, Xavier, Aditya Siddhant, Orhan Firat, and Ankur Parikh. 2021. Harnessing multilinguality in unsupervised machine translation for rare languages. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  41. Goel, Vaibhava and William J. Byrne. 2000. Minimum Bayes-risk automatic speech recognition. Computer Speech and Language, 14(2):115–135. https://doi.org/10.1006 /csla.2000.0138 Goldwater, Sharon and David McClosky. 2005. Improving statistical MT through morphological analysis. In Proceedings of Huma… find
  42. Goyal, Vikrant and Dipti Misra Sharma. 2019. The IIIT-H Gujarati-English machine translation system for WMT19. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 191–195. https:// doi.org/10.18653/v1/W19-5316 Grönroos, Stig Arne, Sami Virpioj… find
  43. Gu, Jiatao, Yong Wang, Yun Chen, Victor O. K. Li, and Kyunghyun Cho. 2018. Meta-learning for low-resource neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3622–3631. https://doi.org/10 .18653/v1/D18-1398 Gülçehre, Çaglar,… find our card
  44. Haddow, Barry and Faheem Kirefu. 2020. PMIndia - A collection of parallel corpora of languages of India. CoRR, abs/2001 .09907. find
  45. Hasan, Tahmid, Abhik Bhattacharjee, Kazi Samin, Masum Hasan, Madhusudan Basak, M. Sohel Rahman, and Rifat Shahriyar. 2020. Not low-resource anymore: Aligner ensembling, batch filtering, and new datasets for Bengali-English machine translation. In Proceedings of the 2020 Conference on Empirical Metho… find our card
  46. He, Di, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. 2016. Dual learning for machine translation. In Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc. find
  47. He, Junxian, Jiatao Gu, Jiajun Shen, and Marc’aurelio Ranzato. 2019. Revisiting Self-Training for neural sequence generation. In Proceedings of the 7th International Conference on Learning Representations. find
  48. He, Xuanli, Gholamreza Haffari, and Mohammad Norouzi. 2020. Dynamic programming encoding for subword segmentation in neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3042–3051. https://doi.org/10.18653/v1/2020.acl -main.275… ↗
  49. Hieber, Felix, Tobias Domhan, Michael Denkowski, and David Vilar. 2020. Sockeye 2: A toolkit for neural machine translation. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 457–458. find
  50. Hoang, Vu Cong Duy, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018. Iterative back-translation for neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18–24. https://doi .org/10.18653/v1/W18-2703 Hokamp, Chris, John Glover, and… find our card
  51. Huszar, Ferenc. 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? CoRR, abs/1511.05101. Jean, Sébastien, Orhan Firat, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. Montreal neural machine translation systems for WMT’15. In Proceedings of the Tent… find
  52. Swanson, Tommi A. Pirinen, Irene Tang, and Hèctor Alòs i Font. 2021. Recent advances in Apertium, a free/open-source rule-based machine translation platform for low-resource languages. Machine Translation, pages 1–28. https://doi.org /10.1007/s10590-021-09260-6 Kim, Yoon and Alexander M. Rush. 201… ↗ our card
  53. Kim, Yunsu, Petre Petrov, Pavel Petrushkov, Shahram Khadivi, and Hermann Ney. 2019. Pivot-based transfer learning for neural machine translation between Non-English languages. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Co… find
  54. Klein, Guillaume, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, pages 67–72. https://doi .org/10.18653/v1… find our card
  55. Knowles, Rebecca, Darlene Stewart, Samuel Larkin, and Patrick Littell. 2020b. NRC systems for the 2020 Inuktitut-English news translation task. In Proceedings of the Fifth Conference on Machine Translation, pages 156–170. Ko, Wei Jen, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman… find
  56. Koehn, Philipp. 2020. Neural Machine Translation. Cambridge University Press. https://doi.org/10.1017/9781108608480 Koehn, Philipp, Vishrav Chaudhary, Ahmed El-Kishky, Naman Goyal, Peng-Jen Chen, and Francisco Guzmán. 2020. Findings of the WMT 2020 shared task on parallel corpus filtering and align… ↗
  57. Koehn, Philipp and Hieu Hoang. 2007. Factored translation models. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 868–876. Koehn, Philipp, Hieu Hoang, Alexandra Birch, Chris Callison-Burc… find
  58. Koehn, Philipp, Huda Khayrallah, Kenneth Heafield, and Mikel L. Forcada. 2018. Findings of the WMT 2018 shared task on parallel corpus filtering. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 726–739. https://doi.org/10.18653/v1/W18 -6453 Koehn, Philipp and… ↗
  59. Kudo, Taku. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75. https://doi.org/10.18653/v1/P18-1007 ↗
  60. Kudo, Taku and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71. https://doi.org/10.18653… find
  61. Lake, Brenden M. 2019. Compositional generalization through meta sequence-to-sequence learning. In Advances in Neural Information Processing Systems, volume 32, pages 9791–9801, Curran Associates, Inc. find
  62. Lakew, Surafel Melaku, Aliia Erofeeva, and Marcello Federico. 2018. Neural machine translation into language varieties. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 156–164. https://doi.org/10 .18653/v1/W18-6316 Lample, Guillaume, Alexis Conneau, Ludovic Deno… find
  63. Lample, Guillaume, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018b. Phrase-based & neural unsupervised machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5039–5049. https://doi.org/10 .18653/v1/D18-1549 Lap… find
  64. Lepikhin, Dmitry, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. GShard: Scaling giant models with conditional computation and automatic sharding. In International Conference, Online. find
  65. Li, Bei, Yinqiao Li, Chen Xu, Ye Lin, Jiqiang Liu, Hui Liu, Ziyang Wang, Yuhao Zhang, Nuo Xu, Zeyang Wang, Kai Feng, Hexuan Chen, Tengbo Liu, Yanyang Li, Qiang Wang, Tong Xiao, and Jingbo Zhu. 2019. The NiuTrans machine translation systems for WMT19. In Proceedings of the Fourth Conference on Machin… find
  66. Li, Zuchao, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, and Eiichiro Sumita. 2020. SJTU-NICT’s supervised and unsupervised neural machine translation systems for the WMT20 news translation task. In Proceedings of the Fifth Conference on Machine Translation, pages 218–229. https://doi.org/10.18653… find
  67. Lignos, Constantine. 2010. Learning from unseen data. In Proceedings of the Morpho Challenge 2010 Workshop, pages 35–38. find
  68. Lin, Yu Hsiang, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019. Choosing transfer languages for cross-lingual learning. In Proceedings of the 57th Annual Meeting of… find our card
  69. Luong, Minh Thang, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. In Proceedings of the 4th International Conference on Learning Representations. Ma, Qingsong, Johnny Wei, Ondřej Bojar, and Yvette Graham. 2019. Results of the WMT19 metr… find our card
  70. Martindale, Marianna, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying fluently inadequate output in neural and statistical machine translation. In Proceedings of Machine Translation Summit XVII: Research Track, pages 233–243. find
  71. Mayer, Thomas and Michael Cysouw. 2014. Creating a massively parallel Bible corpus. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3158–3163. Miceli Barone, Antonio Valerio, Jindřich Helcl, Rico Sennrich, Barry Haddow, and Alexandra Birch.… find
  72. Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of International Conference on Learning Representations. find
  73. Moore, Robert C. and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the ACL 2010 Conference Short Papers, pages 220–224. find our card
  74. Mueller, Aaron, Garrett Nicolai, Arya D. McCarthy, Dylan Lewis, Winston Wu, and David Yarowsky. 2020. An analysis of massively multilingual neural machine translation for low-resource languages. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 3710–3718. find
  75. Mukiibi, Jonathan, Babirye Claire, and Nakatumba-Nabende Joyce. 2021. An English-Luganda parallel corpus. https://doi.org/10.5281/zenodo .4764039 Muller, Benjamin, Antonios Anastasopoulos, Benoı̂t Sagot, and Djamé Seddah. 2021. When being unseen from mBERT is just the beginning: Handling new langua… ↗ our card
  76. Müller, Mathias, Annette Rios, and Rico Sennrich. 2020. Domain robustness in neural machine translation. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 151–164. find
  77. Murthy, Rudra, Anoop Kunchukuttan, and Pushpak Bhattacharyya. 2019. Addressing word-order divergence in multilingual neural machine translation for extremely low resource languages. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:… find
  78. Nakazawa, Toshiaki, Katsuhito Sudoh, Shohei Higashiyama, Chenchen Ding, Raj Dabre, Hideya Mino, Isao Goto, Win Pa Pa, Anoop Kunchukuttan, and Sadao Kurohashi. 2018. Overview of the 5th workshop on Asian translation. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Comp… find
  79. Neishi, Masato, Jin Sakuma, Satoshi Tohda, Shonosuke Ishiwatari, Naoki Yoshinaga, and Masashi Toyoda. 2017. A bag of useful tricks for practical neural machine translation: Embedding layer initialization and large batch size. In Proceedings of the 4th Workshop on Asian Translation (WAT2017), pages 9… find
  80. The IWSLT 2018 evaluation campaign. In Proceedings of the 15th International Workshop on Spoken Language Translation, pages 2–5. find
  81. Niehues, Jan and Eunah Cho. 2017. Exploiting linguistic resources for neural machine translation using multi-task learning. In Proceedings of the Second Conference on Machine Translation, pages 80–89. https://doi.org/10.18653 /v1/W17-4708 Niu, Xing, Weijia Xu, and Marine Carpuat. 2019. Bi-directiona… find
  82. Onome Orife, Iroro Fred, Julia Kreutzer, Blessing Sibanda, Daniel Whitenack, Kathleen Siminyu, Laura Martinus, Jamiil Toure Ali, Jade Abbott, Vukosi Marivate, Salomon Kabongo, Musie Meressa, Espoir Murhabazi, Orevaoghene Ahia, Elan van Biljon, Arshath Ramkilowan, Adewale Akinfaderin, Alp Ktem, Wole… find
  83. Ortega, John E., Richard Castro Mamani, and Kyunghyun Cho. 2020. Neural machine translation with a polysynthetic low resource language. Machine Translation, 34(4):325–346. https://doi.org/10.1007 /s10590-020-09255-9 Ortiz Suárez, Pedro Javier, Benoı̂t Sagot, and Laurent Romary. 2019. Asynchronous p… find
  84. Ott, Myle, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In International Conference on Machine Learning. https://doi.org/10.18653/v1 /W18-6301 Ott, Myle, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grang… find our card
  85. Pavlick, Ellie, Matt Post, Ann Irvine, Dmitry Kachaev, and Chris Callison-Burch. 2014. The language demographics of Amazon Mechanical Turk. Transactions of the Association for Computational Linguistics, 2(Feb):79–92. https://doi.org/10 .1162/tacl a 00167 find
  86. Pennington, Jeffrey, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. https://doi.org/10.3115/v1/D14-1162 ↗
  87. Peters, Matthew, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo… find
  88. Platanios, Emmanouil Antonios, Mrinmaya Sachan, Graham Neubig, and Tom Mitchell. 2018. Contextual parameter generation for universal neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 425–435. https://doi.org/10.18653/v1/D18-1… ↗
  89. Post, Matt, Chris Callison-Burch, and Miles Osborne. 2012. Constructing parallel corpora for six Indian languages via crowdsourcing. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 401–409. find
  90. Provilkov, Ivan, Dmitrii Emelianenko, and Elena Voita. 2020. BPE-dropout: Simple and effective subword regularization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1882–1892. https://doi.org/10 .18653/v1/2020.acl-main.170 Qi, Ye, Devendra Sachan,… find our card
  91. Raffel, Colin, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  92. Ramachandran, Prajit, Peter Liu, and Quoc Le. 2017. Unsupervised pretraining for sequence to sequence learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 383–391. https://doi .org/10.18653/v1/D17-1039 Ramesh, Gowtham, Sumanth Doddapaneni, Aravin… find
  93. Ranathunga, Surangika, En-Shiun Annie Lee, Marjana Prifti Skenduli, Ravi Shekhar, Mehreen Alam, and Rishemjit Kaur. 2021. Neural machine translation for low-resource languages: A survey. CoRR, abs/2106.15115. — [arXiv:2106.15115](https://arxiv.org/abs/2106.15115) find
  94. Ranzato, Marc’Aurelio, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico. find our card
  95. Raunak, Vikas, Arul Menezes, and Marcin Junczys-Dowmunt. 2021. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1172–1183. https:/… find our card
  96. Rezende, Danilo Jimenez, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 1278–1286. Rios, Ann… find our card
  97. Saleva, Jonne and Constantine Lignos. 2021. The effectiveness of morphology-aware segmentation in low-resource neural machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 164–174. https:/… find
  98. Sánchez-Cartagena, Vı́ctor M., Mikel L. Forcada, and Felipe Sánchez-Martı́nez. 2020. A multi-source approach for Breton–French hybrid machine translation. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 61–70. Sánchez-Cartagena, Vı́ctor M.,… find
  99. Santoro, Adam, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with memory-augmented neural networks. In Proceedings of the 33rd International Conference on International Conference on Machine Learning, pages 1842–1850. find
  100. Scherrer, Yves. 2018. The University of Helsinki submissions to the IWSLT 2018 low-resource translation task. In Proceedings of the 15th International Workshop on Spoken Language Translation, pages 83–88. Scherrer, Yves, Stig-Arne Grönroos, and Sami Virpioja. 2020. The University of Helsinki and Aa… find
  101. Schmidhuber, J. 1992. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139. https://doi .org/10.1162/neco.1992.4.1.131 Schulz, Philip, Wilker Aziz, and Trevor Cohn. 2018. A stochastic decoder for neural machine translation. In Proce… find
  102. Sennrich, Rico and Martin Volk. 2011. Iterative, MT-based sentence alignment of parallel texts. In Proceedings of the 18th Nordic Conference of Computational Linguistics (NODALIDA 2011), pages 175–182. find
  103. Sennrich, Rico and Biao Zhang. 2019. Revisiting low-resource neural machine translation: A case study. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 211–221. https://doi.org/10.18653/v1/P19-1021 ↗
  104. Setiawan, Hendra, Matthias Sperber, Udhyakumar Nallasamy, and Matthias Paulik. 2020. Variational neural machine translation with normalizing flows. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7771–7777. https://doi.org/10 .18653/v1/2020.acl-main.… find
  105. Singh, Keshaw. 2020. Adobe AMPS’s submission for very low resource supervised translation task at WMT20. In Proceedings of the Fifth Conference on Machine Translation, pages 1144–1149. find
  106. Singh, Salam Michael, Thoudam Doren Singh, and Sivaji Bandyopadhyay. 2020. The NITS-CNLP system for the unsupervised MT task at WMT 2020. In Proceedings of the Fifth Conference on Machine Translation, pages 1139–1143. find
  107. Siripragada, Shashank, Jerin Philip, Vinay P. Namboodiri, and C. V. Jawahar. 2020. A multilingual parallel corpora collection effort for Indian languages. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 3743–3751. find
  108. Song, Kai, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. 2019b. Code-switching for enhancing NMT with pre-specified translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lon… find
  109. Song, Kaitao, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019a. MASS: Masked sequence to sequence pre-training for language generation. In Proceedings of the 36th International Conference on Machine Learning, pages 5926–5936. find our card
  110. Stahlberg, Felix and Bill Byrne. 2019. On NMT search errors and model errors: Cat got your tongue? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3356–3362. http… find
  111. Tamchyna, Aleš, Marion Weller-Di Marco, and Alexander Fraser. 2017. Modeling target-side inflection in neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 32–42. https://doi.org/10.18653 /v1/W17-4704 Tang, Yuqing, Chau Tran, Xian Li, Peng-Jen Chen, Nama… find our card
  112. Toral, Antonio, Sheila Castilho, Ke Hu, and Andy Way. 2018. Attaining the unattainable? Reassessing claims of human parity in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 113–123. https://doi.org /10.18653/v1/W18-6312 Toral, Antoni… ↗ our card
  113. Tran, Chau, Shruti Bhosale, James Cross, Philipp Koehn, Sergey Edunov, and Angela Fan. 2021. Facebook AI’s WMT21 news translation task submission. In Proceedings of the Sixth Conference on Machine Translation, pages 205–215. find
  114. Uszkoreit, Jakob, Jay Ponte, Ashok Popat, and Moshe Dubiner. 2010. Large scale parallel document mining for machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1101–1109. Varga, D., L. Németh, P. Halácsy, A. Kornai, V. Trón, a… find
  115. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In 31st Conference on Neural Information Processing Systems, pages 6000–6010. Vinyals, Oriol, Charles Blundell, Timothy Lillicrap, koray kavu… find
  116. Wang, Chaojun and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3544–3552. https://doi.org/10.18653/v1/2020.acl -main.326 Wang, Rui, Xu Tan, Renqia… ↗
  117. Williams, Philip, Marcin Chochowski, Pawel Przybysz, Rico Sennrich, Barry Haddow, and Alexandra Birch. 2018. Samsung and University of Edinburgh’s system for the IWSLT 2018 low resource MT task. In Proceedings of the 15th International Workshop on Spoken Language Translation, pages 118–123. find
  118. Williams, Ronald J. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3–4):229–256. https:// doi.org/10.1007/BF00992696 Wiseman, Sam and Alexander M. Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedin… find
  119. Wu, Lijun, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. Beyond error propagation in neural machine translation: Characteristics of language also matter. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3602–3611. https://doi.… ↗
  120. Xia, Yingce, Xu Tan, Fei Tian, Fei Gao, Di He, Weicong Chen, Yang Fan, Linyuan Gong, Yichong Leng, Renqian Luo, Yiren Wang, Lijun Wu, Jinhua Zhu, Tao Qin, and Tie-Yan Liu. 2019. Microsoft Research Asia’s systems for WMT19. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Sha… find
  121. Xu, Nuo, Yinqiao Li, Chen Xu, Yanyang Li, Bei Li, Tong Xiao, and Jingbo Zhu. 2019. Analysis of back-translation methods for low-resource neural machine translation. In Natural Language Processing and Chinese Computing, pages 466–475. https://doi .org/10.1007/978-3-030-32236-6 42 Xu, Weijia, Xing Niu… find
  122. Yang, Jiacheng, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Yong Yu, Weinan Zhang, and Lei Li. 2020. Towards making the most of BERT in neural machine translation. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, pages 9378–9385. find
  123. Yang, Jian, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song, and Furu Wei. 2021. Multilingual machine translation systems from Microsoft for WMT21 shared task. In Proceedings of the Sixth Conference on Machine Translation, pa… find
  124. Zaremoodi, Poorya, Wray Buntine, and Gholamreza Haffari. 2018. Adaptive knowledge sharing in multi-task learning: Improving low-resource neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 656–661. htt… find
  125. Zhang, Yikai, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen. 2020b. Learning with feature-dependent label noise: A progressive approach. In Proceedings of the 8th International Conference on Learning Representations. find
  126. Zhang, Yuhao, Ziyang Wang, Runzhe Cao, Binghao Wei, Weiqiao Shan, Shuhan Zhou, Abudurexiti Reheman, Tao Zhou, Xin Zeng, Laohu Wang, Yongyu Mu, Jingnan Zhang, Xiaoqian Liu, Xuanjun Zhou, Yinqiao Li, Bei Li, Tong Xiao, and Jingbo Zhu. 2020c. The NiuTrans machine translation systems for WMT20. In Proce… find
  127. Zhong, Xing Jie and David Chiang. 2020. Look it up: Bilingual and monolingual dictionaries improve neural machine translation. In Proceedings of the Fifth Conference on Machine Translation, pages 538–549. find
  128. Zoph, Barret, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575. https://doi.org/10.18653/v1/D16-1163 ↗