Elamur Research⌕
☰
home / topics / Evaluation & benchmarks / page 7

📏 Evaluation & benchmarks

684 papers · page 7 of 8 · 526–625 · the most cited first · all topics

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

 Papers (684)

1 … 6 7 8

  1. Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic TranslationWorkshop on NLP for Similar Languages, Varieties and Dialects (VarDial)Abdullah Alabdullah, Lifeng Han, Chenghua Lin2026 · 2 citations
  2. Culture-Aware Machine Translation in Large Language Models: Benchmarking and InvestigationAnnual Meeting of the Association for Computational Linguistics (ACL)Zekun Yuan, Yangfan Ye, Xiaocheng Feng, et al.2026 · 2 citations
  3. MTQE.en-he: Machine Translation Quality Estimation for English-HebrewWorkshop on Language Models for Low-Resource LanguagesAndy Rosenbaum, Assaf Siani, Ilan Kernerman2026 · 2 citations
  4. PHEB: An European Portuguese High School-Level LLM BenchmarkLanguage Resources and Evaluation Conference (LREC)Diogo C. Tavares, Rafael Ferreira, Afonso Simplício, et al.2026 · 2 citations
  5. Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error AnnotationsFindings of the Association for Computational LinguisticsShaomu Tan, Ryosuke Mitani, Ritvik Choudhary, et al.2026 · 2 citations
  6. UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in UrduFindings of the Association for Computational LinguisticsFarah Adeeba, Brian Dillon, Hassan Sajjad, et al.2026 · 2 citations
  7. When Flores Bloomz Wrong: Cross-Direction Contamination in Machine Translation EvaluationConference of the European Chapter of the Association for Computational Linguistics (EACL)David Tan, Pinzhen Chen, Josef van Genabith, et al.2026 · 2 citations
  8. Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen DomainsFindings of the Association for Computational LinguisticsFinn Schmidt, Jan Philip Wahle, Terry Ruas, et al.2026 · 2 citations
  9. AmaSQuAD: A Benchmark for Amharic Extractive Question AnsweringarXiv:2502.02047Nebiyou Daniel Hailemariam, Blessed Guda, Tsegazeab Tefferi2025 · 2 citations
  10. UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via HUMAIN ChatarXiv:2508.17378Omer Nacar2025 · 2 citations
  11. Evaluating the Quality of a Corpus Annotation Scheme Using Pretrained Language ModelsLanguage Resources and Evaluation Conference (LREC)Furkan Akkurt, Onur Gungor, Büşra Marşan, et al.2024 · 2 citations
  12. Evaluation of Semantic Search and its Role in Retrieved-Augmented-Generation (RAG) for Arabic LanguagearXiv:2403.18350Ali Mahboub, Muhy Eddin Za'ter, Bashar Al-Rfooh, et al.2024 · 2 citations
  13. On the Evaluation of Terminology Translation Errors in NMT and PB-SMT in the Legal Domain: a Study on the Translation of Arabic Legal Documents into English and FrenchProceedings of the Workshop on Computational Terminology in NLP and Translation Studies (ConTeNTS) Incorporating the 16th Workshop on Building and Using Comparable Corpora (BUCC)Khadija Ait ElFqih, Johanna Monti2023 · 2 citations
  14. Evaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain AnalysisSagar Indurkhya, Beracah Yankama, Robert C. Berwick2021 · 2 citations
  15. Parser Evaluation Using Derivation Trees: A Complement to evalbSeth Kulick, Ann Bies, Justin Mott, et al.2014 · 2 citations
  16. Ranking Translations using Error Analysis and Quality EstimationMark Fishel2013 · 2 citations
  17. A Bilingual Bimodal Benchmark for Arabic-English NLP across Grammatical Correction, Essay Scoring, Morphological Tagging, and Speech RecognitionLanguage Resources and Evaluation Conference (LREC)Bashar Alhafni, Injy Hamed, Fadhl Eryani, et al.2026 · 1 citation
  18. AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic MemesarXiv:2607.27393Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat, et al.2026 · 1 citation
  19. Anchoring the Judge: Curriculum-Based Adaptation and Reference-Anchored MQM for LLM-Based Machine Translation of an Unseen Low-Resource Language - A Case of NupeWorkshop on Language Models for Low-Resource LanguagesUmar Baba Umar, Sulaimon Adebayo Bashir, Abdulmalik Danlami Mohammed2026 · 1 citation
  20. ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety ClassificationarXiv:2608.01291 · EMNLP 2026 FindingsWajdi Zaghouani, Md. Rafiul Biswas, Kholoud Khalil Aldous, et al.2026 · 1 citation
  21. Biases in Translation: Assessing Opinion Distortion in Machine Translated TextsLanguage Resources and Evaluation Conference (LREC)Nazanin Shafiabadi, François Yvon2026 · 1 citation
  22. BIS Reasoning 1.0: The First Large-Scale Japanese Benchmark for Belief-Inconsistent Syllogistic ReasoningLanguage Resources and Evaluation Conference (LREC)Ha Thanh Nguyen, Hideyuki Tachibana, Chaoran Liu, et al.2026 · 1 citation
  23. EPOP: A Benchmark Corpus for Assessing NLP Models on Structured Information Extraction in Plant HealthLanguage Resources and Evaluation Conference (LREC)Claire Nedellec, Marine Courtin, Xinzhi Yao, et al.2026 · 1 citation
  24. Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered LensFindings of the Association for Computational LinguisticsHellina Hailu Nigatu, Bethelhem Yemane Mamo, Bontu Fufa Balcha, et al.2026 · 1 citation
  25. Evaluation of Adversarial Robustness in Arabic Language ModelsarXiv:2607.25814Anwar Alajmi, Ayed Salman, Imtiaz Ahmad2026 · 1 citation
  26. Gretino: A Greek and Latin Dataset to Benchmark Retrieval Systems in Classical LanguagesLanguage Resources and Evaluation Conference (LREC)Hawau Olamide Toyin, Federico Iezzi, Elia Scapini, et al.2026 · 1 citation
  27. Hurdles of Automatic Metric for Speech Translation EvaluationInternational Conference on Spoken Language Translation (IWSLT)Victor Eugen Zarzu, Vilem Zouhar2026 · 1 citation
  28. Is This Idea Novel? An Automated Benchmark for Judgment of Research IdeasLanguage Resources and Evaluation Conference (LREC)Tim Schopf, Michael Färber2026 · 1 citation
  29. Multi-modal, Multi-task, Multi-criteria Automatic Evaluation with Vision Language ModelsLanguage Resources and Evaluation Conference (LREC)Masanari Oi, Masahiro Kaneko, Naoaki Okazaki, et al.2026 · 1 citation
  30. MUSCAT: MUltilingual, SCientific ConversATion BenchmarkLanguage Resources and Evaluation Conference (LREC)Supriti Sinhamahapatra, Thai-Binh Nguyen, Yiğit Oğuz, et al.2026 · 1 citation
  31. PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine TranslationAnnual Meeting of the Association for Computational Linguistics (ACL)Lorenzo Proietti, Roman Grundkiewicz, Matt Post2026 · 1 citation
  32. Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and DatasetsFindings of the Association for Computational LinguisticsHanna Yukhymenko, Anton Alexandrov, Martin Vechev2026 · 1 citation
  33. Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian LanguagesAnnual Meeting of the Association for Computational Linguistics (ACL)Amir Hossein Yari, Kalmit Kulkarni, Ahmad Raza Khan, et al.2026 · 1 citation
  34. SALT-31: A Machine Translation Benchmark Dataset for 31 Ugandan LanguagesWorkshop on African Natural Language ProcessingSolomon Nsumba, Benjamin Akera, Evelyn Nafula Ouma, et al.2026 · 1 citation
  35. Tie-Calibrated COMETKiwi for Speech Translation Quality Estimation: IWSLT2026 Metrics TrackInternational Conference on Spoken Language Translation (IWSLT)Mubashir Hussain Shah, Aymen Fatima, Kiho Choi, et al.2026 · 1 citation
  36. Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling TasksLanguage Resources and Evaluation Conference (LREC)Elena Alvarez-Mellado, Julio Gonzalo2026 · 1 citation
  37. Uncertainty Quantification for Evaluating Gender Bias in Machine TranslationFindings of the Association for Computational LinguisticsIeva Staliunaite, Julius Cheng, Andreas Vlachos2026 · 1 citation
  38. Word Alignment-Based Evaluation of Uniform Meaning RepresentationsDaniel Zeman, Federica Gamba2026 · 1 citation
  39. XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation MetricsFindings of the Association for Computational LinguisticsJingxuan Liu, Zhi Qu, Jin Tei, et al.2026 · 1 citation
  40. A Comparative Benchmark of a Moroccan Darija Toxicity Detection Model (Typica.ai) and Major LLM-Based Moderation APIs (OpenAI, Mistral, Anthropic)arXiv:2505.04640Hicham Assoudi2025 · 1 citation
  41. Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based BenchmarkDavid M. Smiley2025 · 1 citation
  42. A New Benchmark for Evaluating Automatic Speech Recognition in the Arabic Call DomainarXiv:2403.04280Qusai Abo Obaidah, Muhy Eddin Za'ter, Adnan Jaljuli, et al.2024 · 1 citation
  43. Allen Institute for AI @ SIGTYP 2024 Shared Task on Word Embedding Evaluation for Ancient and Historical LanguagesLester James V. Miranda2024 · 1 citation
  44. Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmenters in Hadith DomainarXiv:2307.09630Huda AlShuhayeb, Behrouz Minaei-Bidgoli, Mohammad E. Shenassa, et al.2023 · 1 citation
  45. OSMAN — A Novel Arabic Readability MetricLanguage Resources and Evaluation Conference (LREC)Mahmoud El-Haj, Paul Rayson2016 · 1 citation
  46. A Benchmark Dataset and Comparative Evaluation of Phonemized and Romanized Urdu for Text-to-SpeechLanguage Resources and Evaluation Conference (LREC)M Kaab Bin Shahid, Muhammed Izharuddin2026
  47. A Comparative Evaluation of Open-Source Models for Russian-Kazakh TranslationWorkshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT)Gleb Shanshin2026
  48. A Fine-Grained Linguistic Evaluation of Low-Resource Luxembourgish–English MTWorkshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT)Nils Rehlinger2026
  49. A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech TranslationInternational Conference on Spoken Language Translation (IWSLT)Yulin Xue, Siqi Ouyang, Lei Li2026
  50. Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech RecognitionarXiv:2609.35564Omid Ghahroodi, Anas Madkoor, Dima Faris Al Saudi, et al.2026
  51. AmharicStoryQA: A Multicultural Story Question Answering Benchmark in AmharicarXiv:2602.02774Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, et al.2026
  52. Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT ModelsLanguage Resources and Evaluation Conference (LREC)Spyridon Mavromatis, Sokratis Sofianopoulos, Prokopis Prokopidis, et al.2026
  53. Arabic ChartSumm: An English-to-Arabic Benchmark for Metadata-to-Text SummarizationLanguage Resources and Evaluation Conference (LREC)Passant Elchafei, Amany Fashwan2026
  54. Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM EvaluationarXiv:2604.03395Leen AlQadi, Ahmed Alzubaidi, Mohammed Alyafeai, et al.2026
  55. Are Language Models Borrowing-Blind? A Multilingual Evaluation of Loanword Identification across 10 LanguagesLanguage Resources and Evaluation Conference (LREC)Merilin Sousa Silva, Sina Ahmadi2026
  56. ASCAT: An Arabic Scientific Corpus and Benchmark for Advanced Translation EvaluationarXiv:2604.00015Serry Sibaee, Khloud Al Jallad, Zineb Yousfi, et al.2026
  57. ASCAT: Arabic Scientific Benchmark for Advanced Translation EvaluationWorkshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLINGSerry Sibaee, Khloud Al Jallad, Zineb Yousfi, et al.2026
  58. Benchmark Data Contamination in Underrepresented Languages: A Comprehensive Analysis Using Brazilian DataLanguage Resources and Evaluation Conference (LREC)Iriedson Souto Maior de Moraes Vilar, David Candeia Maia, João Brunet, et al.2026
  59. Benchmarking Arabic — Russian Machine Translation: A Comparison of Fine-tuned NMT and Few-shot LLMs under Rich Morphology and Low Lexical OverlaparXiv:2609.29559Mullosharaf K. Arabov2026
  60. Beyond BLEU: Linguistic Invisibility and Interactional Repair Sequence in End-to-End Sign Language TranslationProceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in MotionZirui Wang, Mayumi Bono2026
  61. Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language ModelsarXiv:2608.29990Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad, et al.2026
  62. Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex TextRefael Shaked Greenfeld, Reut Tsarfaty2026
  63. Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual AugmentationLanguage Resources and Evaluation Conference (LREC)Kaveh Eskandari Miandoab, Mahammed Kamruzzaman, Arshia Gharooni, et al.2026
  64. Can Video LLMs See Through Illusions? Video-Illusion QA Benchmark DatasetLanguage Resources and Evaluation Conference (LREC)Souto Ohira, Tosho Hirasawa, Mamoru Komachi2026
  65. Chinese Coreference Corpus from Verb-Driven Events and Its Application for LLM Evaluation (动词驱动事件的共指关系中文语料库构建及大模型评测)Journal of Chinese Information ProcessingHaoze Wang (王皓泽), Xin Xin (辛欣)2026
  66. ChineseDevBench: A Chinese Developmental Benchmark for Language DevelopmentWorkshop on Cognitive Modeling and Computational Linguistics (CMCL)Shaonan Wang, Yiwen Wu, Na Li, et al.2026
  67. DAMETA: An LLM Benchmark for Danish Metaphor Interpretation with Systematically Varied DistractorsLanguage Resources and Evaluation Conference (LREC)Nina Skovgaard Schneidermann, Sanni Nimb, Nathalie Carmen Hau Norman, et al.2026
  68. Detecting Latin in Historical Books with Large Language Models: A Multimodal BenchmarkConference of the European Chapter of the Association for Computational Linguistics (EACL)Yu Wu, Ke Shu, Jonas Fischer, et al.2026
  69. Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark SuiteLanguage Resources and Evaluation Conference (LREC)Klaudia Thellmann, Bernhard Stadler, Michael Färber2026
  70. Discourse Structure Variation for Large Language Model Evaluation in Text Simplification (大模型文本简化的篇章结构变化分析与评测)Journal of Chinese Information ProcessingJunhui Zhu (朱君辉), Erhong Yang (杨尔弘), Haonan Zhu (朱浩楠), et al.2026
  71. Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT EvaluationAnnual Meeting of the Association for Computational Linguistics (ACL)Luke Zhang, Justin Vasselli, Aditya Khan, et al.2026
  72. E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference DatasetsIEEE AccessKhloud AL Jallad, Nada Ghneim, Ghaida Rebdawi2026
  73. EduBench: A Portuguese Benchmark for Open-Ended Discursive Question AnsweringLanguage Resources and Evaluation Conference (LREC)Pedro Henrique Paiola, Luís Gabriel Damiati Mendes, Bruno de Oliveira Monchelato, et al.2026
  74. EIFFEL: a novel benchmark to measure bias of English heavy training on French idiomatic expressionsAnnual Meeting of the Association for Computational Linguistics (ACL)Charlotte Noel, Nicholas Asher, Olivier Gouvert, et al.2026
  75. Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural LanguageLanguage Resources and Evaluation Conference (LREC)Yoshiki Tanaka, Ryuichi Uehara, Koji Inoue, et al.2026
  76. eSciBench: An Extensible Scientific PDF Extraction BenchmarkLanguage Resources and Evaluation Conference (LREC)Noah Tremblay Taillon, Phillippe Langlais2026
  77. Evaluating Discriminability of Vision-Language ModelsLanguage Resources and Evaluation Conference (LREC)Masayasu Muraoka, Naoaki Okazaki2026
  78. Evaluating Frontier LLM Translation Capability for LakotaWorkshop on NLP for Indigenous Languages of the Americas (AmericasNLP)Lance Robertson2026
  79. Evaluating Hierarchical Aggregation and LLM-Based Matching for Synset Selection in Ancient GreekWorkshop on Language Technologies for Historical and Ancient Languages @ LRECLuca Brigada Villa, Marco Passarotti, Chiara Zanchi, et al.2026
  80. Evaluating Humanities Theory Alignment in Large Language Models: Incremental Prompting and Statistical AssessmentWorkshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL)Axel Pichler, Janis Pagel2026
  81. Evaluating Latin and Ancient Greek Sentence Alignment through Parallel Sentence MiningWorkshop on Natural Language Processing for Digital HumanitiesSebastian Reichbauer, Shu Okabe, Alexander Fraser2026
  82. Evaluating LLM Style Transfer Through Readability-Based Age AssessmentsInternational Workshop on Spoken Dialogue System TechnologyMaria Di Maro, Antonio Origlia, Leonilda Bilo, et al.2026
  83. Evaluating Multimodal Large Language Model Narrative Interpretation through the Lens of Appraisal TheoryLanguage Resources and Evaluation Conference (LREC)Jayant Teotia, Xiaowei Wang, Xulang Zhang, et al.2026
  84. Evaluating Native-Speaker Preferences on Machine Translation and Post-Edits for Five African LanguagesWorkshop on African Natural Language ProcessingHiba El Oirghi, Tajuddeen Gwadabe, Marine Carpuat2026
  85. Evaluating Small Language Models for English-to-Portuguese Translation: Impact of Model Scale and QuantizationInternational Conference on Computational Processing of PortugueseGustavo Lopes Tamiosso, Rafael Oleques Nunes, Dennis Giovani Balreira2026
  86. Evaluating the Adaptability of Large Language Models to Linguistic VariationLanguage Resources and Evaluation Conference (LREC)Ziyan Xu, Marina Seghier, Alice Millour, et al.2026
  87. Evaluating the Effect of Question Wording Variations on Answer Consistency in Large Language ModelsLanguage Resources and Evaluation Conference (LREC)Junya Takayama, Masaya Ohagi, Tomoya Mizumoto, et al.2026
  88. Evaluating the Impact of Source Diversity for RAG in Historical ResearchLanguage Resources and Evaluation Conference (LREC)Ruhi Mahadeshwar, Andreas van Cranenburgh, Tommaso Caselli, et al.2026
  89. Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in FrenchClinical Natural Language Processing WorkshopMarco Naguib, Christel Gérardin, Victor Beaucoté, et al.2026
  90. Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?Language Resources and Evaluation Conference (LREC)Prateek Kumar Rajput, Yewei Song, Iyiola Emmanuel Olatunji, et al.2026
  91. Evaluation of Document-Level Text Simplification in JapaneseLanguage Resources and Evaluation Conference (LREC)Iori Yamashita, Hikari Tanaka, Hajime Kiyama, et al.2026
  92. Evaluation of Multilingual Text Simplification for the Mental Health Domain: Exploring Small Language ModelsBioNLPOlga Pelloni, Sandra Just, Lars Bongo2026
  93. Evaluation of Paralinguistic-Aware Spoken Dialogue Systems using Next-Utterance ClassificationAnnual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)Kouki Miyazawa, Yoshinao Sato2026
  94. Extending Czech Aspect-Based Sentiment Analysis with Opinion Terms: Dataset and LLM BenchmarksLanguage Resources and Evaluation Conference (LREC)Jakub Šmíd, Pavel Priban, Pavel Kral2026
  95. Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation MetricarXiv:2608.20964Sami Shames El Deen, Mariette Awad2026
  96. Frame2KG: A Benchmark and Evaluation Toolkit for Interpretable Frame-to-Graph GenerationLanguage Resources and Evaluation Conference (LREC)Lewis N. Watson, Carl Strathearn, Kenny Mitchell, et al.2026
  97. From Rosetta to Match-Up: A Paired Corpus of Linguistic Puzzles with Human and LLM BenchmarksLanguage Resources and Evaluation Conference (LREC)Neh Majmudar, Anne Huang, Jinfan Frank Hu, et al.2026
  98. GreekCommonGen: A Benchmark for Evaluating Generative Commonsense Reasoning in GreekSIGUL Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered LanguagesAristotelis Stamopoulos, Dimitrios Galanis2026
  99. Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful HumorarXiv:2603.17759Ahmed Sharshar, Hosam Elgendy, Saad El Dine Ahmed, et al.2026
  100. HERMeS: Human Evaluation & Ranking of MultiplE SystemsAnnual Conference of the European Association for Machine Translation (EAMT)Rex Vanhorn2026

1 … 6 7 8

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/7/