Elamur Research
☰
home / papers / A Challenge Set Approach to Evaluating Machine… / cited by

Who cites «A Challenge Set Approach to Evaluating Machine Translation»

111 papers of our library cite it, newest first: found in their bibliographies, which we read ourselves. ← the card

Pages: 1 · 2

  1. A Fine-Grained Linguistic Evaluation of Low-Resource Luxembourgish–English MT — 2026 · Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)
  2. Augmenting Text to Increase Translation Difficulty — 2026 · Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)
  3. Benchmarking Multilingual LLM Translation Accuracy for Fuzhounese — 2026 · Proceedings of the SIGUL 2026 Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages
  4. Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations? — 2026 · arXiv:2608.08283
  5. Generating Difficult-to-Translate Texts — 2026 · Proceedings of the First Workshop on Multilingual Multicultural Evaluation
  6. Memorisation Meets Compositionality in Natural Language Processing — 2026 · Proceedings of The Big Picture v2: Crafting a Research Narrative
  7. Round-Trip Translation Reveals What Frontier Multilingual Benchmarks Miss — 2026 · arXiv:2604.12911
  8. The Validity of Coreference-based Evaluations of Natural Language Understanding — 2026 · arXiv:2602.16200
  9. A Large-Scale Benchmark for Vietnamese Sentence Paraphrases — 2025 · Findings of the Association for Computational Linguistics: NAACL 2025
  10. A Testset for Context-Aware LLM Translation in Korean-to-English Discourse Level Translation — 2025 · Proceedings of the 31st International Conference on Computational Linguistics
  11. Enhancing Neural Machine Translation Through Target Language Data: A $k$NN-LM Approach for Domain Adaptation — 2025 · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  12. From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling — 2025 · Findings of the Association for Computational Linguistics: EMNLP 2025
  13. GENDEROUS: Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset — 2025 · Proceedings of the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
  14. MSLC25: Metric Performance on Low-Quality Machine Translation, Empty Strings, and Language Variants — 2025 · Proceedings of the Tenth Conference on Machine Translation
  15. Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets — 2025 · Computational Linguistics
  16. Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model — 2025 · Findings of the Association for Computational Linguistics: ACL 2025
  17. Using LLMs to Advance Idiom Corpus Construction — 2025 · Proceedings of the 21st Workshop on Multiword Expressions (MWE 2025)
  18. A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models — 2024 · Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
  19. Context-Aware Neural Machine Translation Models Analysis And Evaluation Through Attention — 2024
  20. Exploring NMT Explainability for Translators Using NMT Visualising Tools — 2024 · Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1)
  21. GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection? — 2024 · Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
  22. Quality and Quantity of Machine Translation References for Automatic Metrics — 2024 · Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024
  23. Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution — 2024 · Proceedings of the 28th Conference on Computational Natural Language Learning
  24. Vers la traduction automatique des néologismes scientifiques — 2024 · Actes de la 31ème Conférence sur le Traitement Automatique des Langues Naturelles, volume 1 : articles longs et prises de position
  25. Évaluation de la Similarité Textuelle : Entre Sémantique et Surface dans les Représentations Neuronales — 2024 · Actes de la 31ème Conférence sur le Traitement Automatique des Langues Naturelles, volume 1 : articles longs et prises de position
  26. A Multilingual Evaluation of NER Robustness to Adversarial Inputs — 2023 · Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023)
  27. A Research-Based Guide for the Creation and Deployment of a Low-Resource Machine Translation System — 2023 · Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing
  28. ACES: Translation Accuracy Challenge Sets at WMT 2023 — 2023 · Proceedings of the Eighth Conference on Machine Translation
  29. Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement — 2023 · Transactions of the Association for Computational Linguistics
  30. Cheating to Identify Hard Problems for Neural Machine Translation — 2023 · Findings of the Association for Computational Linguistics: EACL 2023
  31. Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus — 2023 · arXiv:2305.11142 · ACL 2023
  32. Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels — 2023 · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  33. Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases — 2023 · Findings of the Association for Computational Linguistics: ACL 2023
  34. Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT? — 2023 · Proceedings of the Eighth Conference on Machine Translation
  35. Long to reign over us: A Case Study of Machine Translation and a New Monarch — 2023 · Findings of the Association for Computational Linguistics: ACL 2023
  36. Quality Fit for Purpose: Building Business Critical Errors Test Suites — 2023 · Proceedings of the 24th Annual Conference of the European Association for Machine Translation
  37. Translating Dislocations or Parentheticals : Investigating the Role of Prosodic Boundaries for Spoken Language Translation of French into English — 2023 · Proceedings of Machine Translation Summit XIX, Vol. 2: Users Track
  38. Utiliser les syntagmes nominaux complexes anglais pour évaluer la robustesse des systèmes de traduction anglais-français en langue de spécialité — 2023 · Actes de CORIA-TALN 2023. Actes des 16e Rencontres Jeunes Chercheurs en RI (RJCRI) et 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL)
  39. A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output — 2022 · Proceedings of the Thirteenth Language Resources and Evaluation Conference
  40. ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics — our card · 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)
  41. Biais de genre dans un système de traduction automatique neuronale : une étude des mécanismes de transfert cross-langue [Gender bias in a neural machine translation system: a study of crosslingual transfer mechanisms] — 2022
  42. BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation — 2022 · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
  43. Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation — 2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  44. Food for Thought: How can we exploit contextual embeddings in the translation of idiomatic expressions? — 2022 · Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)
  45. HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models — 2022 · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
  46. Investigating Failures of Automatic Translation in the Case of Unambiguous Gender — 2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
  47. Linguistically Motivated Evaluation of the 2022 State-of-the-art Machine Translation Systems for Three Language Directions — our card · 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)
  48. Linguistically-Motivated Yorùbá-English Machine Translation — 2022 · Proceedings of the 29th International Conference on Computational Linguistics
  49. Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets — 2022 · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
  50. PreQuEL: Quality Estimation of Machine Translation Outputs in Advance — 2022 · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
  51. Test Suite Evaluation: Morphological Challenges and Pronoun Translation — 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)
  52. Toward a Test Set of Dislocations in Persian for Neural Machine Translation — 2022 · Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-located with ICNLSP 2022
  53. Biais de genre dans un système de traduction automatiqueneuronale : une étude préliminaire (Gender Bias in Neural Translation : a preliminary study ) — 2021 · Actes de la 28e Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 : conférence principale
  54. CausaLM: Causal Model Explanation Through Counterfactual Language Models — 2021 · Computational Linguistics
  55. Comparing Knowledge-Intensive and Data-Intensive Models for English Resource Semantic Parsing — 2021 · Computational Linguistics
  56. Contextualized Embeddings Encode Monolingual and Cross-lingual Knowledge of Idiomaticity — 2021 · Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)
  57. Does BERT Understand Idioms? A Probing-Based Empirical Study of BERT Encodings of Idioms — 2021 · Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
  58. HateCheck: Functional Tests for Hate Speech Detection Models — 2021 · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
  59. Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German — 2021 · Proceedings of the Sixth Conference on Machine Translation
  60. NoiseQA: Challenge Set Evaluation for User-Centric Question Answering — 2021 · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
  61. On nature and causes of observed MT errors — 2021 · Proceedings of Machine Translation Summit XVIII: Research Track
  62. Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance? — 2021 · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
  63. Sometimes We Want Ungrammatical Translations — 2021 · Findings of the Association for Computational Linguistics: EMNLP 2021
  64. Tribrid: Stance Classification with Neural Inconsistency Detection — 2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
  65. What Can a Generative Language Model Answer About a Passage? — 2021 · Proceedings of the 3rd Workshop on Machine Reading for Question Answering
  66. An Error-based Investigation of Statistical and Neural Machine Translation Performance on Hindi-to-Tamil and English-to-Tamil — 2020 · Proceedings of the 7th Workshop on Asian Translation
  67. Cross-Linguistic Syntactic Evaluation of Word Prediction Models — 2020 · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
  68. Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale — 2020 · Proceedings of the 28th International Conference on Computational Linguistics
  69. Disambiguation of Potentially Idiomatic Expressions with Contextual Embeddings — 2020 · Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons
  70. Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation — 2020 · Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
  71. Evaluating Models' Local Decision Boundaries via Contrast Sets — our card · 2020 · Findings of the Association for Computational Linguistics: EMNLP 2020
  72. Fine-grained linguistic evaluation for state-of-the-art Machine Translation — 2020 · Proceedings of the Fifth Conference on Machine Translation
  73. No more rage against the machine: how the corpus-based identification of machine-translationese can lead to student empowerment — 2020 · The Journal of Specialised Translation (JoSTrans)
  74. Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English — 2020 · Proceedings of the 28th International Conference on Computational Linguistics
  75. PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents — 2020 · Proceedings of the 28th International Conference on Computational Linguistics
  76. Quantitative Analysis of Post-Editing Effort Indicators for NMT — 2020 · Proceedings of the 22nd Annual Conference of the European Association for Machine Translation
  77. Stance Detection Benchmark: How Robust Is Your Stance Detection? — 2020 · arXiv:2001.01565
  78. Analysis Methods in Neural Language Processing: A Survey — 2019 · Transactions of the Association for Computational Linguistics
  79. Automatically Extracting Challenge Sets for Non-Local Phenomena in Neural Machine Translation — 2019 · Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)
  80. Context in Neural Machine Translation: A Review of Models and Evaluations — 2019 · arXiv:1901.09115
  81. Does NMT make a difference when post-editing closely related languages? The case of Spanish-Catalan — 2019 · Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks
  82. Evaluating Gender Bias in Machine Translation — 2019 · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
  83. Evaluating adversarial attacks against multiple fact verification systems — 2019 · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
  84. Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation — 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
  85. Human Evaluation of Neural Machine Translation: The Case of Deep Learning — 2019 · Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019)
  86. Neural Machine Translation: A Review and Survey — 2019 · arXiv:1912.02047
  87. Non-Parametric Adaptation for Neural Machine Translation — 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
  88. Semantic Neural Machine Translation Using AMR — our card · 2019 · Transactions of the Association for Computational Linguistics
  89. The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation — 2019 · Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)
  90. compare-mt: A Tool for Holistic Comparison of Language Generation Systems — 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)
  91. A fine-grained error analysis of NMT, SMT and RBMT output for English-to-Dutch — 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
  92. Analyzing Uncertainty in Neural Machine Translation — 2018 · arXiv:1803.00047
  93. Automatic Evaluation of English-to-Korean and Korean-to-English Neural Machine Translation Systems by Linguistic Test Points — 2018 · Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation
  94. Construction of a Multilingual Corpus Annotated with Translation Relations — 2018 · Proceedings of the First Workshop on Linguistic Resources for Natural Language Processing
  95. Detecting context-dependent sentences in parallel corpora — 2018 · Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN
  96. Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method — 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
  97. Examining the Tip of the Iceberg: A Data Set for Idiom Translation — 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)
  98. Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks — 2018 · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
  99. Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically motivated Test Suite — 2018 · Proceedings of the AMTA 2018 Workshop on Translation Quality Estimation and Automatic Post-Editing
  100. MTNT: A Testbed for Machine Translation of Noisy Text — 2018 · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Pages: 1 · 2