Elamur Research⌕
☰
home / papers / 2026 / Context-8: A Data Set for Evaluating Context Sensitivity in Machine Translation

Context-8: A Data Set for Evaluating Context Sensitivity in Machine Translation

Summary

Context is widely seen as essential for good translation, but MT research usually treats it narrowly as surrounding text. Existing context-aware test sets also tend to target one phenomenon each, such as pronouns, gender or formality. The authors surveyed 69 papers containing 53 distinct definitions of context and drew on communication models like Hymes' SPEAKING. From this they built a taxonomy with three linguistic and five non-linguistic context categories; the non-linguistic ones include Setting and Scene and Truth Value. They then created Context-8, an English-to-Japanese benchmark organised into groups. Items in a group share one translation "FOCUS" but differ in the surrounding material, so the correct Japanese rendering of that focus shifts. Items are partly hand-crafted and partly collected from the web. The released dataset is the main contribution. The paper also runs illustrative tests with GPT-5.2 and FOCUS-level COMET, but those results are not in the extracted text. The work offers a first broad diagnostic for context sensitivity in MT and a reusable recipe for other language pairs.

Key points

  • First release of Context-8: 130 groups, 533 examples, 1102 sentences (English→Japanese), available at github.com/utlis/context-8.
  • The taxonomy has 3 linguistic and 5 non-linguistic categories. Category labels are non-exclusive, so a single example can carry several codes (e.g., A1 and A2).
  • The data is distributed as one JSON file. Each item has group_id, sid, st (source), ref (Japanese reference), context_impact (accuracy vs. fluency) and context_categories.
  • Evaluation setup: GPT-5.2 with an empty system prompt, a short translation-only user prompt and default API settings. FOCUS spans are located with SimAlign alignment, corrected manually and then scored with COMET.
  • The authors also propose a lightweight workflow that combines hand-crafted and web-sourced items, so similar datasets can be built for other language pairs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3cawdf257c7e

@inproceedings{wang2026context,
  title = {{Context-8: A Data Set for Evaluating Context Sensitivity in Machine Translation}},
  author = {Wang, Dongyue and Kageura, Kyo},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3cawdf257c7e},
  url = {https://doi.org/10.63317/3cawdf257c7e}
}

References

120 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agrawal, R., Turchi, M., and Negri, M. (2018). Contextual handling in neural machine translation: look behind, ahead and on both sides. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation (EAMT 2018), pp. 31-40, Alicante, Spain, May. European Association for Machine Translation. find
  2. Agrawal, S., Zhou, C., Lewis, M., Zettlemoyer, L. and Ghazvininejad, M. (2023). In-Context Examples Selection for Machine Translation. In Proceedings of the Association for Computational Linguistics: ACL 2023, pp. 8857-8873. find our card
  3. Al-Sulaimaan, M. M. D., and Khoshaba, L. M. (2018). Context as a basis for understanding pragmatic ambiguity with reference to Arabic. International Journal of English Literature and Social Sciences, 3(5), pp. 768-775. https://doi.org/10.22161/ijels.3.5.12 ↗
  4. Akman, V., Bouquet, P., Thomason, R., and Young, R. A. (Eds.). (2001). Modeling and using context: CONTEXT 2001. Heidelberg, Germany: Springer. find
  5. Ahmed, M., Ouda, A., Abusharkh, M., Kohli, S., and Rai, K. (2023). An optimized approach to translate technical patents from English to Japanese using machine translation models. Applied Sciences, 13(12), https://doi.org/10.3390/app13127126 ↗
  6. Asher, R. E., and Simpson, J. M. Y. (Eds.). (1994). The encyclopedia of language and linguistics (1st ed.). Tokyo, Japan: Pergamon Press. find
  7. Bahdanau, D., Cho, K., and Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR), San Diego, USA, May. ↗ our card
  8. Baker, K. L., Franz, A. M., and Jordan, P. W. (2001). Coping with ambiguity in knowledge-based natural language analysis [Technical report]. Center for Machine Translation and Department of Philosophy, Carnegie Mellon University. find
  9. Bateson, G. (1972). Steps to an ecology of mind. New York, NY: Chandler. find
  10. Bawden, R., Sennrich, R., Birch, A., and Haddow, B. (2018). Evaluating discourse phenomena in neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1304-1313. New Orleans, LA, June. Association for Computational Linguistics. find our card
  11. Bawden, R., Rosset, S., Lavergne, T., and Bilinski, E. (2019). DiaBLa: A corpus of bilingual spontaneous written dialogues for machine translation. arXiv. https://arxiv.org/abs/1905.13354 ↗
  12. Bazire, M., and Brézillon, P. (2005). Understanding context before using it. In Modeling and using context: 5th International and Interdisciplinary Conference, CONTEXT 2005, pp. 29-40. Paris, France. find
  13. Bell, R. T. (1991). Translation and translating: Theory and practice. London: Longman. find
  14. Berlo, D. K. (1960). The process of communication: An introduction to theory and practice. New York, NY: Holt, Rinehart and Winston. find
  15. Berry, D. M., Kamsties, E., and Krieger, M. M. (2003). From contract drafting to software specification: Linguistic sources of ambiguity (Version 1.0) [Technical report]. University of Waterloo. find
  16. Bloomfield, L. (1953). Language. London, UK: George Allen & Unwin Ltd. find
  17. Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Jimeno Yepes, A., Koehn, P., Logacheva, V., Monz, C., Negri, M., Névéol, A., Neves, M., Popel, M., Post, M., Rubino, R., Scarton, C., Specia, L., Turchi, M., Verspoor, K., and Zampieri, M. (2016). Findings of the 2016 Conference on Machine Translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pp. 131-198. Berlin, Germany, August. Association for Computational Linguistics. find our card
  18. Brézillon, P. (2013). Context-based development of experience bases. In Modeling and Using Context (CONTEXT 2013), LNAI 8175, pp. 87-7126. 100. Berlin/Heidelberg: Springer. find
  19. Buck, D. (2016). Examples of genres. Essentials for ENGL-121. Howard Community College (Pressbooks). Accessed 21 February 2026. find
  20. Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. (2007). (Meta-) Evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pp. 136-158. Prague, Czech Republic, June. Association for Computational Linguistics. find our card
  21. Carpuat, M. and Wu, D. (2007). Context-dependent phrasal translation lexicons for statistical machine translation. In Proceedings of Machine Translation Summit XI: Papers, pp. 73-80, Copenhagen, Denmark, September. find
  22. Catford, J. C. (1965). A linguistic theory of translation: An essay in applied linguistics. Oxford, UK: Oxford University Press. find
  23. Conia, S., Lee, D., Li, M., Minhas, U. F., Potdar, S., and Li, Y. (2024). Towards cross-cultural machine translation with retrieval-augmented generation from multilingual knowledge graphs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), pp. 16343-16360, Miami, Florida, USA, November. Association for Computational Linguistics. find
  24. Crystal, D. (1991). A dictionary of linguistics and phonetics (3rd ed.). Oxford, UK; Cambridge, USA: Blackwell. find
  25. Danks, J. H., Shreve, G. M., Fountain, S. B., and McBeath, M. K. (Eds.). (1997). Cognitive processes in translation and interpreting. Thousand Oaks, CA: Sage Publications. find
  26. Dey, A. K., and Abowd, G. D. (2000). Towards a better understanding of context and context-awareness. In Proceedings of the PrCHI 2000 Workshop on the What, Who, Where, When and How of Context-Awareness, pp. 304-307, The Hague, Netherlands. find
  27. Duranti, A., and Goodwin, C. (Eds.). (1992). Rethinking context: Language as an interactive phenomenon. Cambridge, UK: Cambridge University Press. find
  28. Eisele, A., and Chen, Y. (2010). MultiUN: A multilingual corpus from United Nation documents. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), pp. 2868-2872, Valletta, Malta, May. European Language Resources Association (ELRA). find our card
  29. España-Bonet, C., Stiller, J., and Henning, S. (2018). M1.2 – Corpora for the machine translation engines. In Proceedings of Computer Science, Linguistics, pp. 1-11. find
  30. Fernandes, P., Yin, K., Neubig, G., and Martins, A. F. T. (2021). Measuring and increasing context usage in context-aware machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6467-6478. Online, August. Association for Computational Linguistics. find
  31. Fillmore, C. J. (1982). Frame semantics. In Linguistics in the morning calm, pp. 111-137. Seoul, South Korea: Hanshin. find
  32. Finkbeiner, R., Meibauer, J., and Schumacher, P. B. (Eds.). (2012). What is a context? Linguistic approaches and challenges. Amsterdam, Netherlands: John Benjamins. ↗
  33. Gentzler, E. (1993). Contemporary translation theories. London: Routledge. find
  34. Goffman, E. (1974). Frame analysis: An essay on the organization of experience. New York: Harper & Row. find
  35. Goto, I., Kato, N., Uratani, N., and Ehara, T. (2003). Transliteration considering context information based on the maximum entropy method. In Proceedings of Machine Translation Summit IX: Papers, pp. 125-132, New Orleans, LA, September. find
  36. Gutt, E.A. (1991). Translation and relevance: Cognition and context. Manchester: St Jerome. find
  37. Halliday, M. A. K. (1985). An introduction to functional grammar. London, England: Edward Arnold. find
  38. Halliday, M. A. K., and Hasan, R. (1989). Language, context and text: Aspects of language in a social-semiotic perspective. Oxford, UK: Oxford University Press. find
  39. Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W., Li, M., Liu, S., Liu, T.-Y., Luo, R., Menezes, A., Qin, T., Seide, F., Tan, X., Tian, F., Wu, L., Wu, S., Xia, Y., Zhang, D., Zhang, Z., and Zhou, M. (2018). Achieving human parity on automatic Chinese to English news translation. arXiv. https://arxiv.org/abs/1803.05567 ↗ our card
  40. Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y. J., Afify, M., and Hassan Awadalla, H. (2023). How good are GPT models at machine translation? A comprehensive evaluation. arXiv preprint arXiv:2302.09210. ↗
  41. House, J. (1997). Translation quality assessment: A model revisited. Tübingen, Germany: Gunter Narr Verlag. find
  42. Hu, X., and Wan, X. (2023). Exploring discourse structure in document-level machine translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13889-13902. Singapore, December. Association for Computational Linguistics. ↗
  43. Huang, E., Socher, R., Manning, C., and Ng, A. (2012). Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), Volume 1: Long Papers, pp. 873-882, Jeju, Korea, July. Association for Computational Linguistics. find
  44. Hurford, J. R., and Heasley, B. (1983). Semantics: A coursebook. Cambridge, UK: Cambridge University Press. find
  45. Hurford, J. R., Heasley, B., and Smith, M. B. (2007). Semantics: A coursebook. Cambridge, UK: Cambridge University Press. find
  46. Hwang, Y., Kim, Y., and Jung, K. (2021). Context-aware neural machine translation for Korean honorific expressions. Electronics, 10(13):1589. ↗
  47. Hymes, D. (1967). Models of interaction of language and social setting. Journal of Social Issues, 23(2), pp. 8-28. find
  48. Hymes, D. H. (1974). Foundations in sociolinguistics: An ethnographic approach. Philadelphia, USA: University of Pennsylvania Press. find
  49. Jalili Sabet, M., Dufter, P., Yvon, F. and Schütze, H. (2020). SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings. In Proceedings of the Association for Computational Linguistics: EMNLP 2020, pp. 1627-1643. ↗
  50. Jaszczolt, K. M. (2012). Context: Gricean intentions vs. two-dimensional semantics. In R. Finkbeiner, J. Meibauer, and P. B. Schumacher (Eds.), What is a context? Linguistic approaches and challenges. Amsterdam, Netherlands: John Benjamins. find
  51. Jean, S., Lauly, S., Firat, O., and Cho, K. (2017). Does neural machine translation benefit from larger context? arXiv preprint arXiv:1704.05135. ↗
  52. Jiang, Y. E., Liu, T., Ma, S., Zhang, D., Sachan, M., and Cotterell, R. (2023). Discourse-centric evaluation of document-level machine translation with a new densely annotated parallel corpus of novels. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7853-7872. Toronto, Canada, July. Association for Computational Linguistics. find
  53. Joos, M. (1972). Semantic axiom number one. Language, 48, pp. 257-265. ↗
  54. Kang, X., Zhao, Y., Zhang, J., and Zong, C. (2020). Dynamic context selection for document-level neural machine translation via reinforcement learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), pp. 2242-2254, Online, November. Association for Computational Linguistics. find
  55. Khandelwal, U., Fan, A., Jurafsky, D., Zettlemoyer, L., and Lewis, M. (2021). Nearest neighbor machine translation. arXiv preprint arXiv:2010.00710. ↗
  56. Kim, D., Baek, Y., Yang, S., and Choo, J. (2023). Towards formality-aware neural machine translation by leveraging context information. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7384-7392, Singapore, December. Association for Computational Linguistics. find
  57. Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pp. 79-86. Phuket, Thailand, September. Asia-Pacific Association for Machine Translation. find
  58. Kudo, K., Deguchi, H., Morishita, M., Fujii, R., Ito, T., Ozaki, S., Natsumi, K., Sato, K., Yano, K., Takahashi, R., Kimura, S., Hara, T., Sakai, Y., and Suzuki, J. (2024). Document-level translation with LLM reranking: Team-J at WMT 2024 general translation task. In Proceedings of the Ninth Conference on Machine Translation, pp. 210-226. Miami, Florida, USA, November. Association for Computational Linguistics. find
  59. Leech, G. (1983). Principles of pragmatics. London, England; New York, USA: Longman. find
  60. Levinson, S. C. (1983). Pragmatics. Cambridge, UK: Cambridge University Press. find
  61. Lindley, D. (1964). The editor’s problem. In W. Arrowsmith and R. Shattuck (Eds.), The Craft and Context of Translation: A Critical Symposium, pp. 158-163, Garden City, NY. Anchor Books/Doubleday. find
  62. Lindstrom, L. (1992). Context contests: Debatable truth statements on Tanna (Vanuatu). In R. Finkbeiner, J. Meibauer, and P. B. Schumacher (Eds.), What is a context? Linguistic approaches and challenges. Amsterdam, Netherlands: John Benjamins. find
  63. Liu, S., and Zhang, X. (2020). Corpora for document-level neural machine translation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 3775-3781. Marseille, France, May. European Language Resources Association (ELRA). find
  64. Lyons, J. (1968). Introduction to theoretical linguistics. London, UK: Cambridge University Press. ↗
  65. Lyons, J. (1977). Semantics (Vols. 1 & 2). Cambridge, England: Cambridge University Press. find
  66. Lyons, J. (1995). Linguistic Semantics: An Introduction. London, UK: Cambridge University Press. find
  67. Malinowski, B. (1923). The problem of meaning in primitive languages. In C. K. Ogden and I. A. Richards (Eds.), The meaning of meaning. London, UK: K. Paul, Trench, Trubner. find
  68. Maruf, S., Martins, A. F. T., and Haffari, G. (2019). Selective attention for context-aware neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), Volume 1 (Long and Short Papers), pp. 3092-3102, Minneapolis, Minnesota, USA, June. Association for Computational Linguistics. find
  69. Maruf, S., Saleh, F., and Haffari, G. (2021). A survey on document-level neural machine translation: Methods and evaluation. ACM Computing Surveys, 54(2), pp. 1-36. ↗
  70. Mey, J. L. (1993). Pragmatics: An introduction. Malden, Mass: Blackwell. find
  71. Miller, C., Tzoukermann, E., Doyon, J., and Mallard, E. (2021). Corpus creation and evaluation for speech-to-text and speech translation. In Proceedings of Machine Translation Summit XVIII: Users and Providers Track, pp. 44-53. Virtual, August. European Association for Machine Translation (EAMT). find
  72. Nadejde, M., Currey, A., Hsu, B., Niu, X., Federico, M., and Dinu, G. (2022). CoCoA-MT: A dataset and benchmark for contrastive controlled MT with application to formality. In Findings of the Association for Computational Linguistics: NAACL 2022, pp. 616-632. Seattle, United States, July. Association for Computational Linguistics. find
  73. Nakhlé, M., Dinarelli, M., Qader, R., Esperança-Rodier, E., and Blanchon, H. (2025). DOLFIN – Document-level financial test set for machine translation. arXiv. https://arxiv.org/abs/2502.03053 ↗
  74. Newmark, P. (1981). Approaches to translation. Oxford: Pergamon. find
  75. Nida, E. A. (1964). Toward a science of translating: With special reference to principles and procedures involved in Bible translating. Leiden, Netherlands: E. J. Brill. find
  76. Nida, E. A. (1969). The theory and practice of translation. Leiden: E.J. Brill. find
  77. Nida, E. A. (2001). Contexts in translating. Amsterdam, Netherlands: John Benjamins. ↗
  78. Oaks, D. D. (2010). Structural ambiguity in English: An applied grammatical inventory. London, UK: Continuum. find
  79. Ochs, E. (1979). Introduction: What child language can contribute to pragmatics. In E. Ochs and B. B. Schieffelin (Eds.), Developmental Pragmatics. New York, USA: Academic Press. find
  80. Ott, N., Ziai, R., and Meurers, D. (2012). Creation and analysis of a reading comprehension exercise corpus: Towards evaluating meaning in context. In T. Schmidt et al. (Eds.), Multilingual corpora and multilingual corpus analysis (Hamburg Studies on Multilingualism, Book 14), pp. 47-69. Amsterdam, Netherlands: John Benjamins. ↗
  81. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311-318. Philadelphia, Pennsylvania, USA, July. Association for Computational Linguistics. find our card
  82. Partee, B. H. (1970). Opacity, coreference, and pronouns. Synthese, 21(3-4), pp. 359-385. https://doi.org/10.1007/BF00484805 ↗
  83. Pięta, H., Bueno Maia, R. and Torres-Simón, E. (2023). Indirect translation explained. London: Routledge. ↗
  84. Poyatos, F. (1997). Nonverbal communication and translation: New perspectives and challenges in literature, interpretation, and the media. Amsterdam: John Benjamins. find
  85. Pym, A. (2014). Exploring translation theories. London: Routledge. ↗
  86. Rauf, S. A., and Yvon, F. (2020). Document-level contexts for neural machine translation. [Research Report] 2020-003, LIMSI-CNRS. https://hal.science/hal-03687190v2 ↗
  87. Rei, R., Stewart, C., Farinha, A. C., and Lavie, A. (2020). COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685-2702. Online, November. Association for Computational Linguistics. find our card
  88. Richards, J. C., and Schmidt, R. (2010). Longman dictionary of language teaching and applied linguistics (4th ed.). Harlow, England; Tokyo, Japan: Longman. find
  89. Rivera-Trigueros, I., and Olvera-Lobo, M.-D. (2021). Building a corpus for corporate websites machine-translation evaluation: A step-by-step methodological approach. In Proceedings of the Translation and Interpreting Technology Online Conference, pp. 93-101. Online: INCOMA Ltd. find
  90. Ryan, N. S., Pascoe, J., and Morse, D. R. (1999). Enhanced reality fieldwork: The context-aware archaeological assistant. In L. Dingwall, S. Exon, V. Gaffney, S. Laflin, and M. van Leusen (Eds.), Archaeology in the Age of the Internet: CAA97. Computer Applications and Quantitative Methods in Archaeology. Proceedings of the 25th Anniversary Conference, University of Birmingham, April 1997 (BAR International Series 750), pp. 269-274, Oxford, UK. Archaeopress. ↗
  91. Saluja, A., Lane, I., and Zhang, Y. (2011). Context-aware language modeling for conversational speech translation. In Proceedings of Machine Translation Summit XIII: Papers, Xiamen, China, September. find
  92. Scarton, C., and Specia, L. (2016). A reading-comprehension corpus for machine-translation evaluation. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 16), pp. 3652-3658. Portorož, Slovenia, May. European Language Resources Association (ELRA). find
  93. Sharma, S., Dey, M., and Sinha, K. (2022). How sensitive are translation systems to extra contexts? Mitigating gender bias in neural machine translation models through relevant contexts. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 1968-1984, Abu Dhabi, United Arab Emirates, December. Association for Computational Linguistics. find
  94. Shimazu, S., Takase, S., Nakazawa, T., and Okazaki, N. (2020). Evaluation dataset for zero pronoun in Japanese-to-English translation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 3630-3634. Marseille, France. find
  95. Siripragada, S., Philip, J., Namboodiri, V. P., & Jawahar, C. V. (2020). A multilingual parallel-corpora collection effort for Indian languages. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 3743-3751. Marseille, France, May. European Language Resources Association (ELRA). ↗
  96. Sperber, D., and Wilson, D. (1986). Relevance: Communication and cognition. Oxford, UK: Blackwell. find
  97. Steinberger, R., Pouliquen, B., Widiger, A., Ignat, C., Erjavec, T., Tufiş, D., and Varga, D. (2006). The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06). Genoa, Italy, May. European Language Resources Association (ELRA). find our card
  98. Swinney, D. A. (1979). Lexical access during sentence comprehension: (Re)consideration of context effects. Journal of Verbal Learning and Verbal Behavior, 18(6), pp. 645-659. https://doi.org/10.1016/S0022-5371(79)90355-4 ↗
  99. Tan, X., Zhang, L., Xiong, D., and Zhou, G. (2019). Hierarchical modeling of global context for document-level neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1576-1585. Hong Kong, China, November. Association for Computational Linguistics. find
  100. Tiedemann, J. and Scherrer, Y. (2017). Neural machine translation with extended context. In Proceedings of the Third Workshop on Discourse in Machine Translation, pp. 82-92, Copenhagen, Denmark, September. Association for Computational Linguistics. find our card
  101. van Dijk, T. A. (1977). Text and context: Explorations in the semantics and pragmatics of discourse. London, UK; New York, USA: Longman. find
  102. van Dijk, T. A. (2006). Discourse, context and cognition. Discourse Studies, 8 (1):159-177. ↗
  103. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30, pp. 5998-6008, Long Beach, CA, USA, December. Curran Associates, Inc. find
  104. Vincent, S., Flynn, R., and Scarton, C. (2023). MTCue: learning zero-shot control of extra-textual attributes by leveraging unstructured context in neural machine translation. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 8210-8226, Toronto, Canada, July. Association for Computational Linguistics. find
  105. Vincent, S., Prescott, C., Bayliss, C., Oakley, C., and Scarton, C. (2024). A case study on contextual machine translation in a professional scenario of subtitling. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pp. 561-572, Sheffield, UK, June. European Association for Machine Translation (EAMT). ↗
  106. Voita, E., Serdyukov, P., Sennrich, R., and Titov, I. (2018). Context-aware neural machine translation learns anaphora resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Volume 1: Long Papers, pp. 1264-1274, Melbourne, Australia, July. Association for Computational Linguistics. find our card
  107. Voita, E., Sennrich, R., and Titov, I. (2019). When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 1198-1212, Florence, Italy, July. Association for Computational Linguistics. find our card
  108. Wang, D., Fan, K., Chen, B., and Xiong, D. (2022). Efficient cluster-based k-nearest-neighbor machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), Volume 1: Long Papers, pp. 2175-2187, Dublin, Ireland, May. Association for Computational Linguistics. ↗
  109. Wang, S., Zhang, G., Wu, H., Loakman, T., Huang, W., and Lin, C. (2024). MMTE: Corpus and metrics for evaluating machine-translation quality of metaphorical language. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 11343-11358. Miami, Florida, USA, November. Association for Computational Linguistics. find
  110. Wasow, T., Perfors, A., and Beaver, D. (2005). The puzzle of ambiguity. In C. O. Orgun & P. Sells (Eds.), Morphology and the web of grammar: Essays in memory of Steven G. Lapointe, pp. 265-282. Stanford, CA: CSLI Publications. find
  111. Widdowson, H. G. (2004). Text, context, pretext: Critical issues in discourse analysis. Oxford, UK: Blackwell. find
  112. Winter, W. (1964). Impossibilities of translation. In W. Arrowsmith and R. Shattuck (Eds.), The Craft and Context of Translation, A Critical Symposium, pp. 93-112, Garden City, NY. Anchor Books/Doubleday. find
  113. Wu, H., Dong, D., Hu, X., Yu, D., He, W., Wu, H., Wang, H., and Liu, T. (2014). Improve statistical machine translation with context-sensitive bilingual semantic embedding model. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp. 142-146, Doha, Qatar, October. Association for Computational Linguistics. find
  114. Xu, M., Wang, L., Wong, D. F., Liu, H., Song, L., Chao, L. S., Shi, S., and Tu, Z. (2022). GuoFeng: A benchmark for zero-pronoun recovery and translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11266-11278. Abu Dhabi, United Arab Emirates, December. Association for Computational Linguistics. find our card
  115. Yao, B., Jiang, M., Bobinac, T., Yang, D., and Hu, J. (2024). Benchmarking machine translation with cultural awareness. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 13078-13096, Miami, Florida, USA, November. Association for Computational Linguistics. find our card
  116. Zeng, J., Su, J., Wen, H., Liu, Y., Xie, J., Yin, Y., and Zhao, J. (2018). Multi-domain neural machine translation with word-level domain context discrimination. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pp. 447-457, Brussels, Belgium, October– November. Association for Computational Linguistics. find
  117. Zhang, Y. and Bach, N. (2009). Virtual babel: towards context-aware machine translation in virtual worlds. In Proceedings of Machine Translation Summit XII: Posters, Ottawa, Canada, August 26-30. ↗
  118. Zhang, L., Zhang, Z., Chen, B., Luo, W., and Si, L. (2022). Context-adaptive document-level neural machine translation. In Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6232-6236. Singapore, May. IEEE. find
  119. Zhang, L., Zhang, T., Zhang, H., Yang, B., Ye, W., and Zhang, S. (2021). Multi-hop transformer for document-level machine translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2021), pp. 3953-3963, Online, June. Association for Computational Linguistics. ↗
  120. Zwicky, A. M., and Sadock, J. M. (1975). Ambiguity tests and how to fail them. In J. P. Kimball (Ed.), Syntax and semantics, Vol. 4, pp. 1-36. New York, NY: Academic Press. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/wang-2026-context-8-data-set-evaluating/