Elamur Research⌕
☰
home / papers / 2024 / 文档级神经机器翻译综述

文档级神经机器翻译综述

Summary

Sentence-level neural machine translation (NMT) handles each sentence of a document separately. As a result, it often produces discourse errors such as inconsistent tense across sentences, wrong pronoun or number agreement, and weak cohesion. The authors, from Soochow University and Huawei's translation center, survey document-level NMT (DNMT), meaning methods that use cross-sentence context from the source text, earlier target translations, or both.

The survey first defines DNMT formally as sentence-level encoder-decoder translation conditioned on extra document context. It then covers:

  • shared tasks and the most widely used document-level corpora, with statistics;
  • evaluation metrics designed for discourse phenomena;
  • mainstream model families, from early RNN context encoders and caches to Transformer-based multi-encoder designs;
  • approaches built on pretrained models.

As a review, its main output is an organized taxonomy and resource overview, not a new system. It closes with open problems and future directions. This makes it a useful entry point for anyone building or evaluating context-aware translation, including for lower-resource language pairs.

Key points

  • Motivating example. A Chinese→English Google Translate output from February 2023 switches between present and past tense across sentences. It also renders plural "tadpoles" as singular, which breaks the matching pronouns.
  • Shared tasks.
    • IWSLT has offered document-level TED data since 2012, but scores it with sentence-level metrics (BLEU, METEOR, TER).
    • WMT added document-level corpora in 2019.
    • WAT has included cross-sentence tasks with human evaluation since 2014.
  • Common corpora.
    • Zh→En: LDC/NIST, about 41k documents and 940k sentences.
    • En→De: IWSLT2017, about 1.7k documents and 206k sentences; News-Commentary v11; Europarl v7, about 118k documents and 1.7M sentences.
    • OpenSubtitles2018 is also widely used, but it has no document boundaries.
  • Domain differences. Subtitles are rich in ellipsis and zero anaphora. TED talks are long and contain distant coreference. News shows a balanced mix of phenomena.
  • Discourse metrics surveyed.
    • Pronoun translation: APT, AutoPRF, CRC.
    • Lexical cohesion and consistency: LC/RC, LTCR, HHI.
    • Connectives: ACT.
    • Composite metrics: BLONDE, TCP.
    • Document-level BLEU is common but largely misses these phenomena.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Topics: Discourse analysis · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.13328/j.cnki.jos.007217

@article{2024,
  title = {{文档级神经机器翻译综述}},
  author = {吕星林 and 李军辉 and 陶仕敏 and 杨浩 and 张民},
  journal = {软件学报 (Journal of Software)},
  year = {2024},
  doi = {10.13328/j.cnki.jos.007217},
  url = {https://doi.org/10.13328/j.cnki.jos.007217}
}

References

124 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In: Ghahramani Z, et al. eds. Proc. of the 27th Int’l Conf. on Neural Information Processing Systems. Cambridge: MIT Press, 2014. 3104–3112. find
  2. Kalchbrenner N, Blunsom P. Recurrent continuous translation models. In: Proc. of the 2013 Conf. on Empirical Methods in Natural Language Processing. Seattle: ACL, 2013. 1700–1709. find
  3. Cho K, van Merrienboer B, Gulcehre G, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing. Doha: ACL, 2014. 1724–1734. [doi: 10.3115… find
  4. Meng FD, Lu ZD, Wang MX, Li H, Jiang WB, Liu Q. Encoding source language with convolutional neural network for machine translation. In: Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int’l Joint Conf. on Natural Language Processing (Vol. 1: Long Papers)… find
  5. Gehring J, Auli M, Grangier D, Yarats D, Dauphin YN. Convolutional sequence to sequence learning. In: Proc. of the 34th Int’l Conf. on Machine Learning. Sydney: JMLR.org, 2017. 1243–1252. find our card
  6. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proc. of the 3rd Int’l Conf. on Learning Representations. San Diego: ICLR, 2015. find our card
  7. Luong T, Pham H, Manning CD. Effective approaches to attention-based neural machine translation. In: Proc. of the 2015 Conf. on Empirical Methods in Natural Language Processing. Lisbon: ACL, 2015. 1412–1421. [doi: 10.18653/v1/D15-1166] find
  8. Yang ZC, Hu ZT, Deng YT, Dyer C, Smola A. Neural machine translation with recurrent attention modeling. In: Proc. of the 15th Conf. of the European Chapter of the Association for Computational Linguistics. Valencia: ACL, 2017. 383–387. find
  9. Ranzato MA, Chopra S, Auli M, Zaremba W. Sequence level training with recurrent neural networks. In: Proc. of the 4th Int’l Conf. on Learning Representations. San Juan: ICLR, 2016. find our card
  10. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010. find
  11. Tiedemann J. Context adaptation in statistical machine translation using models with exponentially decaying cache. In: Proc. of the 2010 Workshop on Domain Adaptation for Natural Language Processing. Uppsala: Association for Computational Linguistics, 2010. 8–15. find
  12. Gong ZX, Zhang M, Zhou GD. Cache-based document-level statistical machine translation. In: Proc. of the 2011 Conf. on Empirical Methods in Natural Language Processing. Edinburgh: ACL, 2011. 909–919. find
  13. Hardmeier C, Nivre J, Tiedemann J. Document-wide decoding for phrase-based statistical machine translation. In: Proc. of the 2012 Joint Conf. on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju Island: Association for Computational Linguistics, 2012.… find
  14. Stymne S, Tiedemann J, Hardmeier C, Nivre J. Statistical machine translation with readability constraints. In: Proc. of the 19th Nordic Conf. of Computational Linguistics. Oslo: Linköping University Electronic Press, 2013. 375–386. find
  15. Xiong DY, Ding Y, Zhang M, Tan CL. Lexical chain based cohesion models for document-level statistical machine translation. In: Proc. of the 2013 Conf. on Empirical Methods in Natural Language Processing. Seattle: ACL, 2013. 1563–1573. find
  16. Pearlmutter BA. Learning state space trajectories in recurrent neural networks. Neural Computation, 1989, 1(2): 263–269. [doi: 10.1162/neco.1989.1.2.263] — [doi:10.1162/neco.1989.1.2.263](https://doi.org/10.1162/neco.1989.1.2.263) find
  17. Wang LY, Tu ZP, Way A, Liu Q. Exploiting cross-sentence context for neural machine translation. In: Proc. of the 2017 Conf. on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017. 2826–2831. [doi: 10.18653/v1/D17-1301] find
  18. Tu ZP, Liu Y, Shi SM, Zhang T. Learning to remember translation history with a continuous cache. Trans. of the Association for Computational Linguistics, 2018, 6: 407–420. [doi: 10.1162/tacl_a_00029] find
  19. Zhang JC, Luan HB, Sun MS, Zhai FF, Xu JF, Zhang M, Liu Y. Improving the transformer translation model with document-level context. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 533–542. [doi: 10.18653/v1/D18-1049] find our card
  20. Tan X, Zhang LY, Xiong DY, Zhou GD. Hierarchical modeling of global context for document-level neural machine translation. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 2019. 1576–1585. [d… find
  21. Yang ZX, Zhang JC, Meng FD, Gu SH, Feng Y, Zhou J. Enhancing context modeling with a query-guided capsule network for document-level translation. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: A… find
  22. Macé V, Servan C. Using whole document context in neural machine translation. In: Proc. of the 16th Int’l Conf. on Spoken Language Translation. Hong Kong: ACL, 2019. find
  23. Kang XM, Zhao Y, Zhang JJ, Zong CQ. Dynamic context selection for document-level neural machine translation via reinforcement learning. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. ACL, 2020. 2242–2254. [doi: 10.18653/v1/2020.emnlp-main.175] find
  24. Xu HF, Xiong DY, Van Genabith J, Liu QH. Efficient context-aware neural machine translation with layer-wise weighting and input-aware gating. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. Yokohama: IJCAI, 2021. 544. find
  25. Fernandes P, Yin K, Neubig G, Martins AFT. Measuring and increasing context usage in context-aware machine translation. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 6467–6478. [doi: 10.… find
  26. Bawden R, Sennrich R, Birch A, Haddow B. Evaluating discourse phenomena in neural machine translation. In: Proc. of the 2018 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: ACL, 2018. 1304–1313. [doi: 10.18653/v1/N18-111… find our card
  27. Maruf S, Haffari G. Document context neural machine translation with memory networks. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne: ACL, 2018. 1275–1284. [doi: 10.18653/v1/P18-1118] find
  28. Zheng ZX, Yue X, Huang SJ, Chen JJ, Birch A. Towards making the most of context in neural machine translation. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. Yokohama: IJCAI, 2021. 551. find
  29. Bao GS, Zhang Y, Teng ZY, Chen BX, Luo WH. G-Transformer for document-level machine translation. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 3442–3455. [doi: 10.18653/v1/2021.acl-long.… find
  30. Sun ZW, Wang MX, Zhou H, Zhao CQ, Huang SJ, Chen JJ, Li L. Rethinking document-level neural machine translation. In: Proc. of the 2022 Findings of the Association for Computational Linguistics. Dublin: ACL, 2022. 3537–3548. [doi: 10.18653/v1/2022.findings-acl.279] find
  31. Kang XM, Zhao Y, Zhang JJ, Zong CQ. Enhancing lexical translation consistency for document-level neural machine translation. ACM Trans. on Asian and Low-resource Language Information Processing, 2021, 21(3): 59. [doi: 10.1145/3485469] — [doi:10.1145/3485469](https://doi.org/10.1145/3485469) find
  32. Lyu XL, Li JH, Gong ZX, Zhang M. Encouraging lexical translation consistency for document-level neural machine translation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. ACL, 2021. 3265–3277. [doi: 10.18653/v1/2021.emnlp-main.262] find
  33. Papineni K, Roukos S, Ward T, Zhu WJ. BLEU: A method for automatic evaluation of machine translation. In: Proc. of the 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia: ACL, 2002. 311–318. [doi: 10.3115/1073083.1073135] find our card
  34. Werlen LM, Belis AP. Validation of an automatic metric for the accuracy of pronoun translation (APT). In: Proc. of the 3rd Workshop on Discourse in Machine Translation. Copenhagen: ACL, 2017. 17–25. [doi: 10.18653/v1/W17-4802] find our card
  35. Jiang YC, Liu TY, Ma SM, Zhang DD, Yang J, Huang HY, Sennrich R, Cotterell R, Sachan M, Zhou M. BlonDe: An automatic evaluation metric for document-level machine translation. In: Proc. of the 2022 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Te… find our card
  36. Maruf S, Saleh F, Haffari G. A survey on document-level neural machine translation: Methods and evaluation. ACM Computing Surveys, 2021, 54(2): 45. [doi: 10.1145/3441691] find
  37. Giménez J, Màrquez L, Comelles E, Castellón I, Arranz V. Document-level automatic MT evaluation based on discourse representations. In: Proc. of the 5th Joint Workshop on Statistical Machine Translation and MetricsMATR. Uppsala: ACL, 2010. 333–338. find
  38. Vela M, Tan LL. Predicting machine translation adequacy with document embeddings. In: Proc. of the 10th Workshop on Statistical Machine Translation. Lisbon: ACL, 2015. 402–410. [doi: 10.18653/v1/W15-3051] find
  39. Junczys-Dowmunt M. Microsoft translator at WMT 2019: Towards large-scale document-level neural machine translation. In: Proc. of the 4th Conf. on Machine Translation. Florence: ACL, 2019. 225–233. [doi: 10.18653/v1/W19-5321] find our card
  40. Rysová K, Rysová M, Musil T, Poláková L, Bojar O. A test suite and manual evaluation of document-level NMT at WMT19. In: Proc. of the 4th Conf. on Machine Translation. Florence: ACL, 2019. 455–463. [doi: 10.18653/v1/W19-5352] find
  41. Kuang SH, Xiong DY. Fusing recency into neural machine translation with an inter-sentence gate model. In: Proc. of the 27th Int’l Conf. on Computational Linguistics. Santa Fe: ACL, 2018. 607–617. find
  42. Kuang SH, Xiong DY, Luo WH, Zhou GD. Modeling coherence for neural machine translation with dynamic and topic caches. In: Proc. of the 27th Int’l Conf. on Computational Linguistics. Santa Fe: ACL, 2018. 596–606. find
  43. Yu L, Sartran L, Stokowiec W, Ling W, Kong LP, Blunsom P, Dyer C. Better document-level machine translation with Bayes’ rule. Trans. of the Association for Computational Linguistics, 2020, 8: 346–360. [doi: 10.1162/tacl_a_00319] find
  44. Xiong H, He ZJ, Wu H, Wang HF. Modeling coherence for discourse neural machine translation. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. Honolulu: AAAI, 2019. 7338–7345. [doi: 10.1609/aaai.v33i01.33017338] — [doi:10.1609/aaai.v33i01.33017338](https://doi.org/10.1609/aaai.v33i01.33017338) find
  45. Tan X, Zhang LY, Zhou GD. Coupling context modeling with zero pronoun recovering for document-level natural language generation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. ACL, 2021. 2530–2540. [doi: 10.18653/v1/2021. emnlp-main.197] find
  46. Yamagishi H, Komachi M. Improving context-aware neural machine translation with target-side context. In: Proc. of the 16th Int’l Conf. of the Pacific Association for Computational Linguistics. Hanoi: PACLING, 2019. 112–122. find
  47. Miculicich L, Ram D, Pappas N, Henderson J. Document-level neural machine translation with hierarchical attention networks. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018. 2947–2954. [doi: 10.18653/v1/D18-1325] find our card
  48. Zhang L, Zhang T, Zhang HB, Yang BS, Ye W, Zhang SK. Multi-hop transformer for document-level machine translation. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics. ACL, 2021. 3953–3963. [doi: 10.18653/v1/2021.naacl-main.309] find
  49. Li B, Liu H, Wang ZY, Jiang YF, Xiao T, Zhu JB, Liu TR, Li CL. Does multi-encoder help? A case study on context-aware neural machine translation. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 3512–3518. [doi: 10.18653/v1/2020.acl-main.322] find
  50. Li YC, Li JH, Jiang J, Tao SM, Yang H, Zhang M. P-Transformer: Towards better document-to-document neural machine translation. IEEE/ACM Trans. on Audio, Speech, and Language Processing, 2023, 31: 3859–3870. [doi: 10.1109/TASLP.2023.3313445] — [doi:10.1109/taslp.2023.3313445](https://doi.org/10.1109/taslp.2023.3313445) find
  51. Maruf S, Martins AF, Haffari G. Selective attention for context-aware neural machine translation. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 3092–3102. [doi: 10.18653/v1/N19-1313] find
  52. Wong KY, Maruf S, Haffari G. Contextual neural machine translation improves translation of cataphoric pronouns. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 5971–5978. [doi: 10.18653/v1/2020.acl-main.530] find
  53. Ma SM, Zhang DD, Zhou M. A simple and effective unified encoder for document-level machine translation. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. ACL, 2020. 3505–3511. [doi: 10.18653/v1/2020.acl-main.321] find
  54. Lei YK, Ren YQ, Xiong DY. CoDoNMT: Modeling cohesion devices for document-level neural machine translation. In: Proc. of the 29th Int’l Conf. on Computational Linguistics. Gyeongju: Int’l Committee on Computational Linguistics, 2022. 5205–5216. find
  55. Yun H, Hwang Y, Jung K. Improving context-aware neural machine translation using self-attentive sentence embedding. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 9498–9506. [doi: 10.1609/aaai.v34i05.6494] — [doi:10.1609/aaai.v34i05.6494](https://doi.org/10.1609/aaai.v34i05.6494) find
  56. Mino H, Ito H, Goto I, Yamada I, Tokunaga T. Effective use of target-side context for neural machine translation. In: Proc. of the 28th Int’l Conf. on Computational Linguistics. Barcelona: International Committee on Computational Linguistics, 2020. 4483–4494. [doi: 10.18653/v1/2020.coling-main.396] find
  57. Chen JX, Li X, Zhang JR, Zhou CL, Cui JW, Wang B, Su JS. Modeling discourse structure for document-level neural machine translation. In: Proc. of the 1st Workshop on Automatic Simultaneous Translation. Seattle: ACL, 2020. 30–36. [doi: 10.18653/v1/2020. autosimtrans-1.5] find
  58. Maruf S, Martins AF, Haffari G. Contextual neural model for translating bilingual multi-speaker conversations. In: Proc. of the 3rd Conf. on Machine Translation: Research Papers. Brussels: ACL, 2018. 101–112. [doi: 10.18653/v1/W18-6311] find
  59. Wang XY, Weston J, Auli M, Jernite Y. Improving conditioning in context-aware sequence to sequence models. arXiv:1911.09728, 2019. — [arXiv:1911.09728](https://arxiv.org/abs/1911.09728) find
  60. Yin K, Fernandes P, Pruthi D, Chaudhary A, Martins AF, Neubig G. Do context-aware translation models pay the right attention? In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 788–801. [doi:… find
  61. Voita E, Serdyukov P, Sennrich R, Titov I. Context-aware neural machine translation learns anaphora resolution. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne: ACL, 2018. 1264–1274. [doi: 10.18653/v1/P18-1117] find our card
  62. Voita E, Sennrich R, Titov I. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 1198–1212. [doi: 10.18653/v1/P1… find our card
  63. Voita E, Sennrich R, Titov I. Context-aware monolingual repair for neural machine translation. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 2019. 877–886. [doi: 10.18653/v1/D19-1081] find
  64. Xu MZ, Li LY, Wong DF, Liu Q, Chao LS. Document graph for neural machine translation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. ACL, 2021. 8435–8448. [doi: 10.18653/v1/2021.emnlp-main.663] find
  65. Snover M, Dorr B, Schwartz R, Micciulla L, Makhoul J. A study of translation edit rate with targeted human annotation. In: Proc. of the 7th Conf. of the Association for Machine Translation in the Americas: Technical Papers. Cambridge: Association for Machine Translation in the Americas, 2006. 223–23… find our card
  66. Banerjee S, Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proc. of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. Ann Arbor: ACL, 2005. 65–72. find our card
  67. Lin CY. ROUGE: A package for automatic evaluation of summaries. In: Proc. of the Text Summarization Branches Out. Barcelona: ACL, 2004. 74–81. find
  68. Cai XY, Xiong DY. A test suite for evaluating discourse phenomena in document-level neural machine translation. In: Proc. of the 2nd Int’l Workshop of Discourse Processing. Suzhou: ACL, 2020. 13–17. find
  69. Guillou L, Hardmeier C, Nakov P, Stymne S, Tiedemann J, Versley Y, Cettolo M, Webber B, Popescu-Belis A. Findings of the 2016 WMT shared task on cross-lingual pronoun prediction. In: Proc. of the 1st Conf. on Machine Translation: Vol. 2, Shared Task Papers. Berlin: ACL, 2016. 525–542. [doi: 10.18653… find
  70. Loáiciga S, Stymne S, Nakov P, Hardmeier C, Tiedemann J, Cettolo M, Versley Y. Findings of the 2017 DiscoMT shared task on cross-lingual pronoun prediction. In: Proc. of the 3rd Workshop on Discourse in Machine Translation. Copenhagen: ACL, 2017. 1–16. [doi: 10.18653/v1/W17-4801] find
  71. Hardmeier C, Federico M. Modelling pronominal anaphora in statistical machine translation. In: Proc. of the 7th Int’l Workshop on Spoken Language Translation: Papers. Paris: IWSLT, 2010. 283–289. find our card
  72. Hardmeier C, Nakov P, Stymne S, Tiedemann J, Versley Y, Cettolo M. Pronoun-focused MT and cross-lingual pronoun prediction: Findings of the 2015 DiscoMT shared task on pronoun translation. In: Proc. of the 2nd Workshop on Discourse in Machine Translation. Lisbon: ACL, 2015. 1–16. [doi: 10.18653/v1/W… find
  73. Guillou L, Hardmeier C. PROTEST: A test suite for evaluating pronouns in machine translation. In: Proc. of the 10th Int’l Conf. on Language Resources and Evaluation. Portorož: European Language Resources Association, 2016. 636–643. find
  74. Müller M, Rios A, Voita E, Sennrich R. A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation. In: Proc. of the 3rd Conf. on Machine Translation: Research Papers. Brussels: ACL, 2018. 61–72. [doi: 10.18653/v1/W18-6307] find our card
  75. Jwalapuram P, Joty S, Temnikova I, Nakov P. Evaluating pronominal anaphora in machine translation: An evaluation measure and a test suite. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing. Hong Kong: ACL, 201… find
  76. Shimazu S, Takase S, Nakazawa T, Okazaki N. Evaluation dataset for zero pronoun in Japanese to English translation. In: Proc. of the 12th Language Resources and Evaluation Conf. Marseille: European Language Resources Association, 2020. 3630–3634. find
  77. Wong BTM, Kit C. Extending machine translation evaluation metrics with lexical cohesion to document level. In: Proc. of the 2012 Joint Conf. on Empirical Methods in Natural Language Processing. Jeju Island: ACL, 2012. 1060–1068. find
  78. Gong ZX, Zhang M, Zhou GD. Document-level machine translation evaluation with gist consistency and text cohesion. In: Proc. of the 2nd Workshop on Discourse in Machine Translation. Lisbon: ACL, 2015. 33–40. [doi: 10.18653/v1/W15-2504] find
  79. Lapata M, Barzilay R. Automatic evaluation of text coherence: Models and representations. In: Proc. of the 19th Int’l Joint Conf. on Artificial Intelligence. Edinburgh: Morgan Kaufmann Publishers Inc., 2005. 1085–1090. find
  80. Liang YL, Meng FD, Chen YF, Xu JA, Zhou J. Modeling bilingual conversational characteristics for neural chat translation. In: Proc. of the 59th Annual Meeting of Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 5711–5724. [doi: 10.18… find
  81. Itagaki M, Aikawa T, He XD. Automatic validation of terminology translation consistenscy with statistical method. In: Proc. of the 2007 Machine Translation Summit XI: Papers. Ottawa: MTSummit, 2007. 269–274. find
  82. Guillou L. Analysing lexical consistency in translation. In: Proc. of the 2013 Workshop on Discourse in Machine Translation. Sofia: ACL, 2013. 10–18. find
  83. Reeder F. Measuring MT adequacy using latent semantic analysis. In: Proc. of the 7th Conf. of the Association for Machine Translation in the Americas: Technical Papers. Cambridge: Association for Machine Translation in the Americas, 2006. 176–184. find
  84. Hajlaoui N, Popescu-Belis A. Assessing the accuracy of discourse connective translations: Validation of an automatic metric. In: Proc. of the 14th Int’l Conf. on Intelligent Text Processing and Computational Linguistics. Karlovasi: Springer, 2013. 236–247. [doi: 10.1007/978-3-642-37256-8_20] — [doi:10.1007/978-3-642-37256-8_20](https://doi.org/10.1007/978-3-642-37256-8_20) find
  85. Tiedemann J, Scherrer Y. Neural machine translation with extended context. In: Proc. of the 3rd Workshop on Discourse in Machine Translation. Copenhagen: ACL, 2017. 82–92. [doi: 10.18653/v1/W17-4811] find our card
  86. Jean S, Lauly S, Firat O, Cho K. Does neural machine translation benefit from larger context? arXiv:1704.05135, 2017. find
  87. Kang XM, Zong CQ. Neural machine translation based on multi-task learning of discourse structure. Ruan Jian Xue Bao/Journal of Software, 2022, 33(10): 3806–3818 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6316.htm [doi: 10.13328/j.cnki. jos.006316] — [doi:10.13328/j.cnki](https://doi.org/10.13328/j.cnki) ↗
  88. Tan X, Zhang LY, Kong F, Zhou GD. Towards discourse-aware document-level neural machine translation. In: Proc. of the 31st Int’l Joint Conf. on Artificial Intelligence. Vienna: IJCAI, 2022. 4383–4389. find
  89. Lin ZH, Feng MW, dos Santos CN, Yu M, Xiang B, Zhou BW, Bengio Y. A structured self-attentive sentence embedding. In: Proc. of the 5th Int’l Conf. on Learning Representations. Toulon: ICLR, 2017. find our card
  90. Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: ACL, 2019. 4171–4186. [doi:… find
  91. Kong F, Ge HZ, Zhou GD. Corpus construction for Chinese zero anaphora from discourse perspective. Ruan Jian Xue Bao/Journal of Software, 2021, 32(12): 3782–3801 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6119.htm [doi: 10.13328/j.cnki. jos.006119] — [doi:10.13328/j.cnki](https://doi.org/10.13328/j.cnki) ↗
  92. Hwang Y, Yun H, Jung K. Contrastive learning for context-aware neural machine translation using coreference information. In: Proc. of the 6th Conf. on Machine Translation. ACL, 2021. 1135–1144. find
  93. Lupo L, Dinarelli M, Besacier L. Divide and rule: Effective pre-training for context-aware multi-encoder translation models. In: Proc. of the 60th Annual Meeting of the Association for Computational Linguistics. Dublin: ACL, 2022. 4557–4572. [doi: 10.18653/v1/2022.acl-long.312] find
  94. Merkel M. Consistency and variation in technical translation: A study of translators’ attitudes. In: Proc. of Unity in Diversity, Translation Studies Conf. 1996. 137–149. find
  95. Carpuat M. One translation per discourse. In: Proc. of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. Boulder: ACL, 2009. 19–27. find
  96. Ture F, Oard DW, Resnik P. Encouraging consistent translation choices. In: Proc. of the 2012 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Montréal: ACL, 2012. 417–426. find
  97. Lyu XL, Li JH, Tao SM, Yang H, Qin Y, Zhang M. Modeling consistency preference via lexical chains for document-level neural machine translation. In: Proc. of the 2022 Conf. on Empirical Methods in Natural Language Processing. Abu Dhabi: ACL, 2022. 6312–6326. [doi: 10.18653/v1/2022.emnlp-main.424] find
  98. Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? In: Proc. of the 27th Int’l Conf. on Neural Information Processing Systems. Montreal: MIT Press, 2014. 3320–3328. find
  99. Liu YH, Gu JT, Goyal N, Li X, Edunov S, Ghazvininejad M, Lewis M, Zettlemoyer L. Multilingual denoising pre-training for neural machine translation. Trans. of the Association for Computational Linguistics, 2020, 8: 726–742. [doi: 10.1162/tacl_a_00343] find
  100. Li LY, Jiang X, Liu Q. Pretrained language models for document-level neural machine translation. arXiv:1911.03110, 2019. — [arXiv:1911.03110](https://arxiv.org/abs/1911.03110) find
  101. Sugiyama A, Yoshinaga N. Context-aware decoder for neural machine translation using a target-side document-level language model. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, 2021. 5781–5791. [doi: 10.186… find
  102. OpenAI. GPT-4 technical report. arXiv:2303.08774, 2023. find our card
  103. Xue LT, Constant N, Roberts A, Kale M, Al-Rfou R, Siddhant A, Barua A, Raffel C. mT5: A massively multilingual pre-trained text-to-text transformer. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, 2021. 483… find
  104. Sanh V, Webson A, Raffel C, et al. Multitask prompted training enables zero-shot task generalization. In: Proc. of the 10th Int’l Conf. on Learning Representations. ICLR, 2022. find our card
  105. Wang LY, Lyu CY, Ji TB, Zhang ZR, Yu D, Shi SM, Tu ZP. Document-Level machine translation with large language models. In: Proc. of the 2023 Conf. on Empirical Methods in Natural Language Processing. Singapore: ACL, 2023. 16646–16661. [doi: 10.18653/v1/2023. emnlp-main.1036] find our card
  106. Karpinska M, Iyyer M. Large language models effectively leverage document-level context for literary translation, but critical errors persist. In: Proc. of the 8th Conf. on Machine Translation. Singapore: ACL, 2023. 419–451. [doi: 10.18653/v1/2023.wmt-1.41] find our card
  107. Zhu JH, Xia YC, Wu LJ, He D, Qin T, Zhou WG, Li HQ, Liu TY. Incorporating BERT into neural machine translation. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: ICLR, 2023. find
  108. Guo ZY, Le Nguyen M. Document-level neural machine translation using BERT as context encoder. In: Proc. of the 1st Conf. of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th Int’l Joint Conf. on Natural Language Processing: Student Research Workshop. Suzhou: ACL,… find
  109. Donato D, Yu L, Dyer C. Diverse pretrained context encodings improve document translation. In: Proc. of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int’l Joint Conf. on Natural Language Processing. ACL, 2021. 1299–1311. [doi: 10.18653/v1/2021.acl-long.104] find
  110. Zhang JQ, Zhao Y, Saleh M, Liu PJ. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In: Proc. of the 37th Int’l Conf. on Machine Learning. ICML, 2020. 11328–11339. find our card
  111. Wu XQ, Xia YC, Zhu JH, Wu LJ, Xie SF, Qin T. A study of BERT for context-aware neural machine translation. Machine Learning, 2022, 111(3): 917–935. [doi: 10.1007/s10994-021-06070-y] — [doi:10.1007/s10994-021-06070-y](https://doi.org/10.1007/s10994-021-06070-y) find
  112. Sennrich R, Haddow B, Birch A. Improving neural machine translation models with monolingual data. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016. 86–96. [doi: 10.18653/v1/P16-1009] find
  113. Zhang JJ, Zong CQ. Exploiting source-side monolingual data in neural machine translation. In: Proc. of the 2016 Conf. on Empirical Methods in Natural Language Processing. Austin: ACL, 2016. 1535–1545. [doi: 10.18653/v1/D16-1160] find our card
  114. Farajian MA, Lopes AV, Martins AFT, Maruf S, Haffari G. Findings of the WMT 2020 shared task on chat translation. In: Proc. of the 5th Conf. on Machine Translation. ACL, 2020. 65–75. find our card
  115. Sulubacak U, Caglayan O, Grönroos SA, Rouhe A, Elliott D, Specia L, Tiedemann J. Multimodal machine translation through visuals and speech. Machine Translation, 2020, 34(2): 97–147. [doi: 10.1007/s10590-020-09250-0] — [doi:10.1007/s10590-020-09250-0](https://doi.org/10.1007/s10590-020-09250-0) find
  116. Bérard A, Pietquin O, Servan C, Besacier L. Listen and translate: A proof of concept for end-to-end speech-to-text translation. In: Proc. of NIPS Workshop on End-to-end Learning for Speech and Audio Processing. Barcelona, 2016. find
  117. Weiss RJ, Chorowski J, Jaitly N, Wu YH, Chen ZF. Sequence-to-sequence models can directly translate foreign speech. In: Proc. of the 18th Annual Conf. of the Int’l Speech Communication Association. Stockholm: Interspeech, 2017. 2625–2629. find
  118. Calixto I, Liu Q, Campbell N. Doubly-attentive decoder for multi-modal neural machine translation. In: Proc. of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver: ACL, 2017. 1913–1924. [doi: 10.18653/v1/P17-1175] — [doi:10.18653/v1/p17-1175](https://doi.org/10.18653/v1/p17-1175) find
  119. Huang PY, Liu F, Shiang SR, Oh J, Dyer C. Attention-based multimodal neural machine translation. In: Proc. of the 1st Conf. on Machine Translation. Berlin: ACL, 2016. 639–645. [doi: 10.18653/v1/W16-2360] find our card
  120. Wang X, Wu JW, Chen JK, Li L, Wang YF, Wang WY. VaTeX: A large-scale, high-quality multilingual dataset for video-and-language research. In: Proc. of the 2019 IEEE/CVF Int’l Conf. on Computer Vision. Seoul: IEEE, 2019. 4580–4590. [doi: 10.1109/ICCV.2019.00468] — [doi:10.1109/iccv.2019.00468](https://doi.org/10.1109/iccv.2019.00468) find
  121. Aharoni R, Johnson M, Firat O. Massively multilingual neural machine translation. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics Human Language Technologies. Minneapolis: ACL, 2019. 3874–3884. [doi: 10.18653/v1/N19-1388] find our card
  122. Dabre R, Chu CH, Kunchukuttan A. A survey of multilingual neural machine translation. ACM Computing Surveys, 2020, 53(5): 99. [doi: 10.1145/3406095] find
  123. Zhang B, Bapna A, Johnson M, Dabirmoghaddam A, Arivazhagan N, Firat O. Multilingual document-level translation enables zero-shot transfer from sentences to documents. In: Proc. of the 60th Annual Meeting of the Association for Computational Linguistics. Dublin: ACL, 2022. 4176–4192. [doi: 10.18653/v… find
  124. Zhu WH, Liu HY, Dong QX, Xu JJ, Huang SJ, Kong LP, Chen JJ, Li L. Multilingual machine translation with large language models: Empirical results and analysis. arXiv:2304.04675, 2023. 附中文参考文献: [87] 亢晓勉, 宗成庆. 基于篇章结构多任务学习的神经机器翻译. 软件学报, 2022, 33(10): 3806–3818. http://www.jos.org.cn/1000-9825/6316.htm [… — [doi:10.13328/j.cnki.jos.006316](https://doi.org/10.13328/j.cnki.jos.006316) ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/lv-2024-document-level-neural/