Elamur Research⌕
☰
home / papers / 2026 / Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model

Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model

Paper · 2026 · Cristian García-Romero, Miquel Esplà-Gomis and Felipe Sánchez-Martínez

Summary

Web-crawled parallel corpora used to train machine translation (MT) systems increasingly contain machine-translated text. Training on too much of it can harm translation quality, so separating human translations (HT) from MT output is a useful cleaning step. The authors frame this as binary classification of a source–target sentence pair. Their method, SMaTD, feeds the source into a frozen multilingual MT model (NLLB) and forces the candidate translation through its decoder. It then takes hidden states from one decoder block, projects them, and passes them to a small transformer encoder classifier. An optional variant, SMaTD+LM, also adds the [CLS] vector from a multilingual encoder LM that was first fine-tuned on the task and then frozen.

The experiments use WMT news data, building on the dataset of Chichirau et al. (2023). The authors extend it with more language pairs and MT systems. SMaTD beats the fine-tuned-encoder state of the art, especially for pairs without English. Because NLLB is not English-centred, the approach suits diverse language pairs. It also needs no access to the system that produced the translation.

Key points

  • Data: Language pairs are de–en, ru–en, en–de, en–ru, de–es and es–de; zh–en was dropped over reliability concerns. Translations come from DeepL, Google, MADLAD-400, Opus-MT and Tower Instruct. Zero-shot tests use M2M-100 as an unseen system and fi–en as an unseen language pair.
  • Splits: WMT 2019 serves as the test set for most pairs; de–es and es–de are tested on WMT 2011/2013.
  • Main gains: Accuracy improves by more than 5 points on non-English pairs. In the zero-shot setting (unseen MT system and language), non-English pairs gain 10–15 points.
  • What drives performance: Larger surrogates (NLLB 3.3B vs 1.3B) perform better, and middle decoder blocks give the most useful representations. Transfer works well across languages but degrades somewhat across MT systems.
  • Resources and limits: Code, datasets and models are released at github.com/transducens/SMaTD. The evaluation covers only the news domain and a single surrogate model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AlicanteAlicante · Spain21 works · 1,098 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Cristian García-Romero, Miquel Esplà-Gomis and Felipe Sánchez-Martínez. 2026. Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.704

@article{garciaromero2026automatic,
  title = {{Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model}},
  author = {Cristian García-Romero, Miquel Esplà-Gomis and Felipe Sánchez-Martínez},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.704},
  url = {https://doi.org/10.1162/tacl.a.704}
}

References

42 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Roee Aharoni, Moshe Koppel, and Yoav Goldberg. 2014. Automatic detection of machine translated text and translation quality estimation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 289–295, Baltimore, Maryland. Association… find
  2. Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and Andre Martins. 2024. Tower: An open multilingual large language model for translation-related tasks. In Fi…
  3. Yuki Arase and Ming Zhou. 2013. Machine translation detection from monolingual web-text. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1597–1607, Sofia, Bulgaria. Association for Computational Linguistics. find
  4. Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019. Massively multilingual neural machine translation in the wild: Findings and challenges. arXiv prepri…
  5. Shivendra Bhardwaj, David Alfonso Hermelo, Phillippe Langlais, Gabriel Bernier-Colborne, Cyril Goutte, and Michel Simard. 2020. Human or neural translation? In Proceedings of the 28th International Conference on Computational Linguistics, pages 6553–6564, Barcelona, Spain (Online). International Com… find
  6. Yongjian Chen, Mireia Farrús, and Antonio Toral. 2025. The potential of speech features to discriminate between original and machine-translated texts. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  7. Malina Chichirau, Rik van Noord, and Antonio Toral. 2023. Automatic discrimination of human and neural machine translation in multilingual scenarios. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 217–226, Tampere, Finland. European Associatio… find
  8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  10. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language P… find
  11. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Michael Auli, and Armand Joulin. 2021. Beyond english-centric multilingual machine tra… find
  12. Yingxue Fu and Mark-Jan Nederhof. 2021. Automatic classification of human translation and machine translation: A study from the perspective of lexical diversity. In Proceedings for the First Workshop on Modelling Translation: Translatology in the Digital Age, pages 91–99, online. Association for Com… find
  13. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116, Florence, Italy. Association for Computat…
  14. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,… — The Llama 3 Herd of Models
  15. Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. MADLAD-400: A multilingual and document-level large audited dataset. In Advances in Neural Information Processing Systems, volume 36, pages 67284–67296. Curran As…
  16. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations.
  17. Yitong Li, Rui Wang, and Hai Zhao. 2015. A machine learning method to distinguish machine translation from human translation. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation: Posters, pages 354–360, Shanghai, China. find
  18. Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger. 2016. Deep networks with stochastic depth. In Computer Vision – ECCV 2016, pages 646–661. Springer International Publishing. find
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  20. Shaoxiong Ji, Zihao Li, Indraneil Paul, Jaakko Paavola, Peiqin Lin, Pinzhen Chen, Dayyán O’Brien, Hengyu Luo, Hinrich Schütze, Jörg Tiedemann, and Barry Haddow. 2024. EMMA-500: Enhancing massively multilingual adaptation of large language models. arXiv preprint arXiv:2409.17892. — [arXiv:2409.17892](https://arxiv.org/abs/2409.17892) find
  21. Rui Jiao, Zonghan Yang, Maosong Sun, and Yang Liu. 2021. Alternated training with synthetic and authentic data for neural machine translation. In Findings of the Association for Compu-Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel,… find
  22. Jiaming Luo, Colin Cherry, and George Foster. 2024. To diverge or not to diverge: A morphosyntactic perspective on machine translation vs human translation. Transactions of the Association for Computational Linguistics, 12:355– 371.
  23. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, pages 24950–24962.
  24. Hoang-Quoc Nguyen-Son and Isao Echizen. 2018. Detecting computer-generated text using fluency and noise features. In International Conference of the Pacific Association for Computational Linguistics, pages 288–300, Singapore. Springer Singapore. find
  25. Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  26. Hoang-Quoc Nguyen-Son, Huy H. Nguyen, Ngoc-Dung T. Tieu, Junichi Yamagishi, and Isao Echizen. 2018. Identifying computer-translated paragraphs using coherence features. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation, Hong Kong. Association for Computation… find
  27. Gema Ramírez-Sánchez, Marta Bañón, Jaume Zaragoza-Bernabeu, and Sergio Ortiz Rojas. 2022. Human evaluation of web-crawled parallel corpora for machine translation. In Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval), pages 32–41, Dublin, Ireland. Association for Computati… find
  28. Hoang-Quoc Nguyen-Son, Tran Thao, Seira Hidano, Ishita Gupta, and Shinsaku Kiyomoto. 2021. Machine translated text detection through text similarity with round-trip translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Huma… find
  29. Nicholas Roberts, Davis Liang, Graham Neubig, and Zachary C. Lipton. 2020. Decoding and diversity in machine translation. In Proceedings of the Resistance AI Workshop at 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, CA. find
  30. Hoang-Quoc Nguyen-Son, Tran Phuong Thao, Seira Hidano, and Shinsaku Kiyomoto. 2019. Detecting machine-translated paragraphs by matching similar words. In International Conference on Computational Linguistics and Intelligent Text Processing, pages 521–532. Springer. find
  31. Hoang-Quoc Nguyen-Son, Ngoc-Dung T. Tieu, Huy H. Nguyen, Junichi Yamagishi, and Isao Echi Zen. 2017. Identifying computer-generated text using statistical analysis. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 1504–1511. IEEE. find
  32. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Gonzalez Mejia, Prangthip Hansanti, John Hof… find
  33. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. AI models collapse when trained on recursively generated data. Nature, 631(8022):755–759. find
  34. Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico. 2024. A shocking amount of the web is machine translated: Insights from multi-way parallelism. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1763–1775, Bangkok, Thailand. Association f… find
  35. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480, Lisboa, Portugal. European Association for Machine Translation. find
  36. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  37. Eva Vanmassenhove, Dimitar Shterionov, and Andy Way. 2019. Lost in translation: Loss and decay of linguistic richness in machine translation. In Proceedings of Machine Translation Summit XVII: Research Track, pages 222–232, Dublin, Ireland. European Association for Machine Translation.
  38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010, Long Beach, USA.
  39. Saranya Venkatraman, Adaku Uchendu, and Dongwon Lee. 2024. GPT-who: An information density-based machine-generated text detector. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 103–115, Mexico City, Mexico. Association for Computational Linguistics. find
  40. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, and Thomas Arnold. 2024. SemEval-2024 task 8: Multidomain, multi-model and multilingual machine-generated text detection. In Proceedings of the 18th Internati… find
  41. Tobias van der Werff, Rik van Noord, and Antonio Toral. 2022. Automatic discrimination of human and neural machine translation: A study with multiple pre-trained models and longer context. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pages 161–170… find
  42. Lijun Wu, Yiren Wang, Yingce Xia, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2019. Exploiting monolingual data at scale for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langua…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/romero-2026-automatic-machine-translation-detection-surrogate/