Elamur Research⌕
☰
home / papers / 2026 / Lost in Translation, and Found: Detecting and Interpreting Translation Effects

Lost in Translation, and Found: Detecting and Interpreting Translation Effects

Paper · 2026 · Shira Wein, Anna Serbina, Jiyuan Ji et al.

Summary

The paper tackles translationese classification: deciding whether an English text was written in English or translated into it. This matters because translated material in training or evaluation data can distort model performance and benchmark results. Humans perform at roughly chance level on this task. The authors used the ENNTT corpus of European Parliament speeches, keeping native-English originals and professional translations balanced across seven source languages. They fine-tuned nine foundation models, including BERT-family encoders, XLNet, two KAN-based architectures and Mamba. They also varied how many consecutive sentences make up each item. Fine-tuned ELECTRA set a new state of the art at above 94% Macro F1. Mid-sized chunks of several sentences gave the strongest results. Interpretability analyses (LIME, SHAP, Integrated Gradients, discourse analysis, linear probing and attention inspection) show that no single cue drives the decision. Instead, many small signals combine, some of which match linguistic theories of translationese. Function words and punctuation are especially prominent. Much of this knowledge is already present in pretrained ELECTRA before any task-specific training. The released classifier can help filter translated text out of datasets.

Key points

  • Data: The ENNTT corpus contains 116,340 original and 116,340 translated English sentences. Each of the seven source languages (Portuguese, Italian, Spanish, French, German, Greek and Dutch) contributes 16,620 sentences to the translated half. Non-native English speech was excluded, and the data was split 70/30 for training and testing.
  • Main result: Fine-tuned ELECTRA reached 94.68% Macro F1. This beats the earlier BERT-based baseline of about 92% and feature-based SVMs at roughly 80%+.
  • Chunk size: Context helps, and chunks of 5–10 sentences work best, with six sentences optimal in this setup. The authors note it is unclear whether this transfers to other domains.
  • Pretrained model: Pretrained ELECTRA with only a trained classification head already achieves 88.83% Macro F1.
  • Probing and attention:
    • Lexical features are most decodable in early layers.
    • Syntactic and discourse features peak around layers 5–7.
    • Fine-tuning mainly changes the final layer.
    • Dodrio attention visualizations show fine-tuned heads linking distant function words and punctuation.
  • Availability: Code and models are on GitHub (ACNLPlab/translationese-classification).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Southern CaliforniaLos Angeles · United States162 works · 35,836 citations🎓University of Massachusetts AmherstAmherst Center · United States106 works · 66,692 citations🎓University of Colorado BoulderBoulder · United States69 works · 51,069 citations🎓University of South FloridaTampa · United States7 works · 33,127 citations🎓Amherst CollegeAmherst Center · United States4 works · 13 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Shira Wein, Anna Serbina, Jiyuan Ji, Nathan Wolf, Jason DeGraaff, Prajakta Kini and Maria Leonor Pacheco. 2026. Lost in Translation, and Found: Detecting and Interpreting Translation Effects. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.781

@inproceedings{wein2026lost,
  title = {{Lost in Translation, and Found: Detecting and Interpreting Translation Effects}},
  author = {Shira Wein, Anna Serbina, Jiyuan Ji, Nathan Wolf, Jason DeGraaff, Prajakta Kini and Maria Leonor Pacheco},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.781},
  url = {https://doi.org/10.18653/v1/2026.acl-long.781}
}

References

43 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644. find
  2. Kwabena Amponsah-Kaakyire, Daria Pylypenko, Cristina España-Bonet, and Josef van Genabith. 2021. Do not rely on relay translations: Multilingual parallel direct Europarl. In Proceedings for the First Workshop on Modelling Translation: Translatology in the Digital Age, pages 1–7, online. Association… find
  3. Kwabena Amponsah-Kaakyire, Daria Pylypenko, Josef Genabith, and Cristina España-Bonet. 2022. Explaining translationese: why are neural classifiers better and what do they learn? In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 281–296, Abu… find
  4. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2020. Translation artifacts in cross-lingual transfer learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7674–7684, Online. Association for Computational Linguistics.
  5. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR. find
  6. Tri Dao and Albert Gu. 2024. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. In International Conference on Machine Learning (ICML). find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  8. Meet Doshi, Raj Dabre, and Pushpak Bhattacharyya. 2024. Pretraining language models using translationese. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5843–5862, Miami, Florida, USA. Association for Computational Linguistics. find
  9. Markus Freitag, Isaac Caswell, and Scott Roy. 2019. APE at scale and its implications on MT evaluation biases. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 34–44, Florence, Italy. Association for Computational Linguistics.
  10. Aditya Nalgunda Ganesh. 2024. Kan-gpt: The pytorch implementation of generative pre-trained transformers (gpts) using kolmogorov-arnold networks (kans) for language modeling. Release 1.0.0, 9th May 2024. find
  11. Yvette Graham, Barry Haddow, and Philipp Koehn. 2020. Statistical power and translationese in machine translation evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 72–81, Online. Association for Computational Linguistics.
  12. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations.
  13. Iustina Ilisei, Diana Inkpen, Gloria Corpas Pastor, and Ruslan Mitkov. 2010. Identification of translationese: A machine learning approach. In Computational Linguistics and Intelligent Text Processing, pages 503– 511, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  14. Rricha Jalota, Koel Chowdhury, Cristina España-Bonet, and Josef van Genabith. 2023. Translating away translationese without parallel data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7086–7100, Singapore. Association for Computational Linguistics. find
  15. Zae Myung Kim, Kwang Lee, Preston Zhu, Vipul Raheja, and Dongyeop Kang. 2024. Threads of subtlety: Detecting machine-generated texts through discourse motifs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5449–5474, Bangkok,… find
  16. J. P. Kincaid, R. P. Fishburne, R. L. Rogers, and B. S. Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical Report 8-75, Naval Technical Training Command Millington TN Research Branch. find
  17. Moshe Koppel and Noam Ordan. 2011. Translationese and its dialects. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1318–1326, Portland, Oregon, USA. Association for Computational Linguistics.
  18. Sergiu Nisioi, Ella Rabinovich, Liviu P. Dinu, and Shuly Wintner. 2016. A corpus of native, non-native and translated texts. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA). find
  19. Maria Kunilovskaya, Koel Dutta Chowdhury, Heike Przybyl, Cristina España-Bonet, and Josef Genabith. 2024. Mitigating translationese with GPT-4: Strategies and performance. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 411–430, Shef… find
  20. OpenAI. 2024. Text generation and prompting. find
  21. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRevi…
  22. Yafu Li, Ronghao Zhang, Zhilin Wang, Huajian Zhang, Leyang Cui, Yongjing Yin, Tong Xiao, and Yue Zhang. 2025. Lost in literalism: How supervised training shapes translationese in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),…
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  24. Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. 2024. Kan: Kolmogorov-arnold networks. Preprint, arXiv:2404.19756. — [arXiv:2404.19756](https://arxiv.org/abs/2404.19756) find
  25. Doreen Osmelak, Koel Dutta Chowdhury, Uliana Sentsova, Cristina España-Bonet, and Josef van Genabith. 2025. Pragextra: A multilingual corpus of pragmatic explicitation in translation. Preprint, arXiv:2511.02721. find
  26. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1135–1144.
  27. Parker Riley, Isaac Caswell, Markus Freitag, and David Grangier. 2020. Translationese as a language in “multilingual” NMT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7737–7746, Online. Association for Computational Linguistics. find
  28. Milan Straka, Jan Hajič, and Jana Straková. 2016. UD-Pipe: Trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4290– 4297,… find
  29. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. CoRR, abs/1703.01365. find
  30. Sonja Tirkkonen-Condit. 2002. Translationese—a myth or an empirical fact?: A study into the linguistic identifiability of translated language. Target. International Journal of Translation Studies, 14(2):207–220. find
  31. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  32. Vered Volansky, Noam Ordan, and Shuly Wintner. 2013. On the features of translationese. Digital Scholarship in the Humanities, 30(1):98–118. find
  33. William C Mann and Sandra A Thompson. 1987. Rhetorical structure theory: A theory of text organization. Technical report, University of Southern California, Information Sciences Institute Los Angeles. find
  34. Jiaan Wang, Fandong Meng, Yunlong Liang, Tingyi Zhang, Jiarong Xu, Zhixu Li, and Jie Zhou. 2023. Understanding translationese in cross-lingual summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3837–3849, Singapore. Association for Computational Linguistic… find
  35. Jingwei Ni, Zhijing Jin, Markus Freitag, Mrinmaya Sachan, and Bernhard Schölkopf. 2022. Original or translated? a causal analysis of the impact of translationese on machine translation performance. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computation… find
  36. Zijie J. Wang, Robert Turko, and Duen Horng Chau. 2021. Dodrio: Exploring transformer models with interactive visualization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System D… find
  37. Shira Wein. 2023. Human raters cannot distinguish English translations from original English texts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12266– 12272, Singapore. Association for Computational Linguistics. find
  38. Shira Wein and Nathan Schneider. 2024. Lost in translationese? reducing translation effect using Abstract Meaning Representation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 753–765, St. Julian’s, Malta…
  39. Xingyi Yang and Xinchao Wang. 2024. Kolmogorov-arnold transformer. Preprint, arXiv:2409.10594. find
  40. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, 32.
  41. Sicheng Yu, Qianru Sun, Hao Zhang, and Jing Jiang. 2022. Translate-train embracing translationese artifacts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 362–370, Dublin, Ireland. Association for Computational Linguistics. find
  42. Mike Zhang and Antonio Toral. 2019. The effect of translationese in machine translation test sets. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 73– 81, Florence, Italy. Association for Computational Linguistics.
  43. Dawei Zhu, Pinzhen Chen, Miaoran Zhang, Barry Haddow, Xiaoyu Shen, and Dietrich Klakow. 2024. Fine-tuning large language models to translate: Will a touch of noisy data in misaligned languages suffice? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wein-2026-lost-translation-found-detecting-interpreting/