Elamur Research⌕
☰
home / topics / Evaluation & benchmarks / page 6

📏 Evaluation & benchmarks

684 papers · page 6 of 8 · 426–525 · the most cited first · all topics

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

 Papers (684)

1 … 5 6 7 8

  1. Evaluating statistical language models as pragmatic reasonersarXiv:2305.01020Benjamin Lipkin, Lionel Wong, Gabriel Grand, et al.2023 · 18 citations
  2. The Iron(ic) Melting Pot: Reviewing Human Evaluation in Humour, Irony and Sarcasm GenerationFindings of the Association for Computational LinguisticsTyler Loakman, Aaron Maladry, Chenghua Lin2023 · 18 citations
  3. GuoFeng: A Benchmark for Zero Pronoun Recovery and TranslationConference on Empirical Methods in Natural Language Processing (EMNLP)Mingzhou Xu, Longyue Wang, Derek F. Wong, et al.2022 · 18 citations
  4. AR-ASAG An ARabic Dataset for Automatic Short Answer Grading EvaluationLanguage Resources and Evaluation Conference (LREC)Leila Ouahrani, Djamal Bennouar2020 · 18 citations
  5. An Evaluation of Parser Robustness for Ungrammatical SentencesHoma B. Hashemi, Rebecca Hwa2016 · 18 citations
  6. Multi-metric optimization for coreference: The UniTN / IITP / Essex submission to the 2011 CONLL Shared TaskConference on Computational Natural Language Learning (CoNLL)Olga Uryupina, Sriparna Saha, Asif Ekbal, et al.2011 · 18 citations
  7. Parser Evaluation and the BNC: Evaluating 4 constituency parsers with 3 metricsJennifer Foster, Josef van Genabith2008 · 18 citations
  8. Ciron: a New Benchmark Dataset for Chinese Irony DetectionLanguage Resources and Evaluation Conference (LREC)Rong Xiang, Xuefeng Gao, Yunfei Long, et al.2020 · 17 citations
  9. The SADID Evaluation Datasets for Low-Resource Spoken Language Machine Translation of Arabic DialectsInternational Conference on Computational Linguistics (COLING)Wael Abid2020 · 17 citations
  10. Comparative Evaluation of Argument Extraction Algorithms in Discourse Relation ParsingInternational Workshop/Conference on Parsing TechnologiesEvgeny Stepanov, Giuseppe Riccardi2013 · 17 citations
  11. Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe RetrievalConference on Empirical Methods in Natural Language Processing (EMNLP)Tianyi Hu, Maria Maistro, Daniel Hershcovich2024 · 16 citations
  12. Opinion Piece: Can we Fix the Scope for Coreference? Problems and Solutions for Benchmarks beyond OntoNotesAmir Zeldes2022 · 16 citations
  13. Linguistically Aware Coreference Evaluation MetricsInternational Joint Conference on Natural Language Processing (IJCNLP)Chen Chen, Vincent Ng2013 · 16 citations
  14. AraSTEM: A Native Arabic Multiple Choice Question Benchmark for Evaluating LLMs Knowledge In STEM SubjectsarXiv:2501.00559Ahmad Mustapha; Hadi Al-Khansa; Hadi Al-Mubasher; Aya Mourad; Ranam Hamoud; Hasan El-Husseini; Marwah Al-Sakkaf;, Mariette Awad2024 · 15 citations
  15. Beyond Correlation: Interpretable Evaluation of Machine Translation MetricsStefano Perrella, Lorenzo Proietti, Pere-Lluís Huguet Cabot, et al.2024 · 15 citations
  16. Evaluating Large Language Models for Health-related Queries with PresuppositionsFindings of the Association for Computational LinguisticsNavreet Kaur, Monojit Choudhury, Danish Pruthi2024 · 15 citations
  17. TurBLiMP: A Turkish Benchmark of Linguistic Minimal PairsConference on Empirical Methods in Natural Language Processing (EMNLP)Ezgi Başar, Francesca Padovani, Jaap Jumelet, et al.2025 · 14 citations
  18. Evaluating Text Generation from Discourse Representation StructuresChunliu Wang, Rik van Noord, Arianna Bisazza, et al.2021 · 14 citations
  19. SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?Conference on Empirical Methods in Natural Language Processing (EMNLP)Senyu Li, Jiayi Wang, Felermino D. M. A. Ali, et al.2025 · 13 citations
  20. Towards a Unified Benchmark for Arabic Pronunciation Assessment: Quranic Recitation as Case StudyInterspeechYassine El Kheir, Omnia Ibrahim, Amit Meghanani, et al.2025 · 13 citations
  21. Open Universal Arabic ASR LeaderboardInterspeechYingzhi Wang, Anas Alhmoud, Muhammad Alqurishi2024 · 13 citations
  22. HeQ: a Large and Diverse Hebrew Reading Comprehension BenchmarkAmir DN Cohen, Hilla Merhav, Yoav Goldberg, et al.2023 · 13 citations
  23. Evaluation of named entity coreferenceWorkshop on Computational Models of Reference, Anaphora and Coreference (CRAC)Oshin Agarwal, Sanjay Subramanian, Ani Nenkova, et al.2019 · 13 citations
  24. Diacritic Annotation in the Arabic Treebank and its Impact on Parser EvaluationMohamed Maamouri, Seth Kulick, Ann Bies2008 · 13 citations
  25. Towards Heterogeneous Automatic MT Error AnalysisJesús Giménez, Lluís Màrquez2008 · 13 citations
  26. Incorporate Semantic Structures into Machine Translation Evaluation via UCCAJin Xu, Yinuo Guo, Junfeng Hu2020 · 12 citations
  27. Evaluating Historical Text Normalization Systems: How Well Do They Generalize?Alexander Robertson, Sharon Goldwater2018 · 12 citations
  28. FrameNet Translation Using Bilingual Dictionaries with Evaluation on the English-French PairLanguage Resources and Evaluation Conference (LREC)Claire Mouton, Gaël de Chalendar, Benoit Richert2010 · 12 citations
  29. MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question AnsweringLanguage Resources and Evaluation Conference (LREC)Wen-wai Yim, Asma Ben Abacha, Zixuan Yu, et al.2026 · 11 citations
  30. DiBiMT: A Gold Evaluation Benchmark for Studying Lexical Ambiguity in Machine TranslationComputational LinguisticsFederico Martelli, Stefano Perrella, Niccolò Campolungo, et al.2025 · 11 citations
  31. Pragmatic Competence Evaluation of Large Language Models for the Korean LanguagePacific Asia Conference on Language, Information and Computation (PACLIC)Dojun Park, Jiwoo Lee, Hyeyun Jeong, et al.2024 · 11 citations
  32. Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human RatingsBlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLPHendrik Schuff, Hsiu-Yu Yang, Heike Adel, et al.2021 · 11 citations
  33. Critical Reflections on Evaluation Practices in Coreference ResolutionProceedings of the 2013 NAACL HLT Student Research WorkshopGordana Ilić Holen2013 · 11 citations
  34. Findings of the Fourth Shared Task on Multilingual Coreference Resolution: Can LLMs Dethrone Traditional Approaches?Michal Novák, Miloslav Konopik, Anna Nedoluzhko, et al.2025 · 10 citations
  35. Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling PerspectiveIan Porada, Alexandra Olteanu, Kaheer Suleman, et al.2024 · 10 citations
  36. Targeted Syntactic Evaluation on the Chomsky HierarchyLanguage Resources and Evaluation Conference (LREC)Taiga Someya, Ryo Yoshida, Yohei Oseki2024 · 10 citations
  37. The Perceptimatic English Benchmark for Speech Perception ModelsarXiv:2005.03418Juliette Millet, Ewan Dunbar2020 · 10 citations
  38. MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMsarXiv:2603.07539Abdessalam Bouchekif; Shahd Gaben; Samer Rashwani; Somaya Eltanbouly; Mutaz Al-Khatib; Heba Sbahi; Mohammed Ghaly;, Emad Mohamed2026 · 9 citations
  39. TriFine: A Large-Scale Dataset of Vision-Audio-Subtitle for Tri-Modal Machine Translation and Benchmark with Fine-Grained Annotated TagsInternational Conference on Computational Linguistics (COLING)Boyu Guan, Yining Zhang, Yang Zhao, et al.2025 · 9 citations
  40. The eBible Corpus: Data and Model Benchmarks for Bible Translation for Low-Resource LanguagesVesa Akerman, David Baines, Damien Daspit, et al.2023 · 9 citations
  41. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language UnderstandingarXiv:2205.10687Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 9 citations
  42. Referenceless Parsing-Based Evaluation of AMR-to-English GenerationEmma Manning, Nathan Schneider2021 · 9 citations
  43. DTED: Evaluation of Machine Translation Structure Using Dependency Parsing and Tree Edit DistanceMartin McCaffery, Mark-Jan Nederhof2016 · 9 citations
  44. A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity UnderstandingLanguage Resources and Evaluation Conference (LREC)Dilara Torunoğlu-Selamet, Doğukan Arslan, Rodrigo Wilkens, et al.2026 · 8 citations
  45. Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious QuestionsProceedings of the AAAI/ACM Conference on AI Ethics and SocietyFarah Atif, Nursultan Askarbekuly, Kareem Darwish, et al.2025 · 8 citations
  46. Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions?Transactions of the Association for Computational Linguistics (TACL)Sweta Agrawal, Amin Farajian, Patrick Fernandes, et al.2024 · 8 citations
  47. Data-driven Parsing Evaluation for Child-Parent InteractionsZoey Liu, Emily Prud'hommeaux2023 · 8 citations
  48. Linguistically Motivated Evaluation of the 2022 State-of-the-art Machine Translation Systems for Three Language DirectionsVivien Macketanz, Shushen Manakhimova, Eleftherios Avramidis, et al.2022 · 8 citations
  49. An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly PapersLanguage Resources and Evaluation Conference (LREC)Kenichi Iwatsuki, Florian Boudin, Akiko Aizawa2020 · 8 citations
  50. Evaluating Word Embeddings on Low-Resource LanguagesNathan Stringham, Mike Izbicki2020 · 8 citations
  51. The Types and Distributions of Errors in a Wide Coverage Surface Realizer EvaluationCharles Callaway2005 · 8 citations
  52. A Comprehensive Evaluation of Large Language Models on Mental Illnesses in Arabic ContextarXiv:2501.06859Noureldin Zahran, Aya E. Fouda, Radwa J. Hanafy, et al.2025 · 7 citations
  53. Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range EntitiesMichal Novák, Miloslav Konopík, Anna Nedoluzhko, et al.2026 · 6 citations
  54. MultiWikiQA: A Reading Comprehension Benchmark in 300+ LanguagesLanguage Resources and Evaluation Conference (LREC)Dan Saattrup Smart2026 · 6 citations
  55. The LSCD Benchmark: a Testbed for Diachronic Word Meaning TasksJoint Conference on Lexical and Computational Semantics (*SEM)Dominik Schlechtweg, Sachin Yadav, Jonas Kuhn, et al.2026 · 6 citations
  56. A Pragmatics-Centered Evaluation Framework for Natural Language UnderstandingLanguage Resources and Evaluation Conference (LREC)Damien Sileo, Philippe Muller, Tim Van de Cruys, et al.2022 · 6 citations
  57. Dependency Parsing Evaluation for Low-resource Spontaneous SpeechZoey Liu, Emily Prud'hommeaux2021 · 6 citations
  58. Evaluating Amharic Machine TranslationarXiv:2003.14386Asmelash Teka Hadgu, Adam Beaudoin, Abel Aregawi2020 · 6 citations
  59. Development and Evaluation of a Personalized Computer-aided Question Generation for English Learners to Improve Proficiency and Correct MistakesarXiv:1808.09732Yi-Ting Huang, Meng Chang Chen, Yeali S. Sun2018 · 6 citations
  60. Evaluation of Finite State Morphological Analyzers Based on Paradigm Extraction from WiktionaryLing Liu, Mans Hulden2017 · 6 citations
  61. In-Context Evaluation of Unsupervised Dialogue Act Models for Tutorial DialogueSIGDIAL ConferenceAysu Ezen-Can, Kristy Boyer2013 · 6 citations
  62. Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation EvaluationAnnual Meeting of the Association for Computational Linguistics (ACL)Yanzhi Tian, Cunxiang Wang, Zeming Liu, et al.2026 · 5 citations
  63. MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and BenchmarkarXiv:2602.01714Mouath Abu-Daoud, Leen Kharouf, Omar El Hajj, et al.2026 · 5 citations
  64. Gender Bias Evaluation in Machine Translation for Amharic, Tigrigna, and Afaan OromooWorkshop on Gender-Inclusive Translation TechnologiesWalelign Sewunetie, Atnafu Tonja, Tadesse Belay, et al.2024 · 5 citations
  65. Overview of the Arabic Sentiment Analysis 2021 Competition at KAUSTarXiv:2109.14456Hind Alamro, Manal Alshehri, Basma Alharbi, et al.2021 · 5 citations
  66. Toward Qualitative Evaluation of Embeddings for Arabic Sentiment AnalysisLanguage Resources and Evaluation Conference (LREC)Amira Barhoumi, Nathalie Camelin, Chafik Aloulou, et al.2020 · 5 citations
  67. Predicting CEFRL levels in learner English on the basis of metrics and full textsarXiv:1806.11099Taylor Arnold, Nicolas Ballier, Thomas Gaillat, et al.2018 · 5 citations
  68. Evaluation Strategies for Computational Construction GrammarsInternational Conference on Computational Linguistics (COLING)Tânia Marques, Katrien Beuls2016 · 5 citations
  69. First Results in a Study Evaluating Pre-annotation and Correction Propagation for Machine-Assisted Syriac Morphological AnalysisPaul Felt, Eric Ringger, Kevin Seppi, et al.2012 · 5 citations
  70. Evaluation Metrics for Knowledge-Based Machine TranslationTeruko Mitamura, Jaime G. Carbonell1994 · 5 citations
  71. Extending ASR Evaluation Resources for Modern Greek DialectsWorkshop on NLP for Similar Languages, Varieties and Dialects (VarDial)Chara Tsoukala, Stavros Bompolas, Antigoni Margariti, et al.2026 · 4 citations
  72. Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual ResultsWorkshop on Generation, Evaluation and Metrics (GEM)Jan-Thorsten Peter, David Vilar, Tobias Domhan, et al.2026 · 4 citations
  73. Multi-Session Client-Centered Treatment Outcome Evaluation in PsychotherapyLanguage Resources and Evaluation Conference (LREC)Hongbin Na, Tao Shen, Shumao Yu, et al.2026 · 4 citations
  74. Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language ModelsBabyLM WorkshopRaha Askari, Sina Zarrieß, Özge Alacam, et al.2025 · 4 citations
  75. Advancing Arabic Sentiment Analysis: ArSen Benchmark and the Improved Fuzzy Deep Hybrid NetworkConference on Computational Natural Language Learning (CoNLL)Yang Fang, Cheng Xu, Shuhao Guan, et al.2024 · 4 citations
  76. How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language ModelsMiriam Cuscito, Alfio Ferrara, Martin Ruskov2024 · 4 citations
  77. A Benchmark Arabic Dataset for Commonsense ExplanationarXiv:2012.10251Saja AL-Tawalbeh, Mohammad AL-Smadi2020 · 4 citations
  78. Evaluating Inter-Annotator Agreement on Historical Spelling NormalizationMarcel Bollmann, Stefanie Dipper, Florian Petran2016 · 4 citations
  79. Towards a Semantic Annotation of English Television News - Building and Evaluating a Constraint Grammar FrameNetPacific Asia Conference on Language, Information and Computation (PACLIC)Eckhard Bick2012 · 4 citations
  80. How Specialized are Specialized Corpora? Behavioral Evaluation of Corpus Representativeness for Maltese.Language Resources and Evaluation Conference (LREC)Jerid Francom, Amy LaCross, Adam Ussishkin2010 · 4 citations
  81. Creation and Evaluation of Extensible Language Resources for MalteseLanguage Resources and Evaluation Conference (LREC)Angelo Dalli2002 · 4 citations
  82. ARB: A Comprehensive Arabic Multimodal Reasoning BenchmarkLanguage Resources and Evaluation Conference (LREC)Sara Ghaboura, Shubham Patle, Ketan More, et al.2026 · 3 citations
  83. Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect VariantsLanguage Resources and Evaluation Conference (LREC)Hunzalah Hassan Bhatti, Firoj Alam2026 · 3 citations
  84. Do Diacritics Matter? Evaluating the Impact of Arabic Diacritics on Tokenization and LLM BenchmarksGo Inoue, Bashar Alhafni, Nizar Habash, et al.2026 · 3 citations
  85. Evaluating Multimodal Large Language Models on Vertically Written Japanese TextLanguage Resources and Evaluation Conference (LREC)Keito Sasagawa, Shuhei Kurita, Daisuke Kawahara2026 · 3 citations
  86. HalluScore: Large Language Model Hallucination Question Answering BenchmarkarXiv:2605.17007Aisha Alansari, Hamzah Luqman2026 · 3 citations
  87. Nahw: A Comprehensive Benchmark of Arabic Grammar Understanding, Error Detection, Correction, and ExplanationHamdy Mubarak, Majd Hawasly, Abubakr Mohamed2026 · 3 citations
  88. A Multi-Task Benchmark for Abusive Language Detection in Low-Resource SettingsConference on Neural Information Processing Systems (NeurIPS)Fitsum Gaim, Hoyun Song, Huije Lee, et al.2025 · 3 citations
  89. BLiMP-IT: Harnessing Automatic Minimal Pair Generation for Italian Language Model EvaluationItalian Conference on Computational LinguisticsMatilde Barbini, Maria Letizia Piccini Bianchessi, Veronica Bressan, et al.2025 · 3 citations
  90. Targeted Syntactic Evaluation for Grammatical Error CorrectionAomi Koyama, Masato Mita, Su-Youn Yoon, et al.2025 · 3 citations
  91. Cyber Risks of Machine Translation Critical Errors : Arabic Mental Health Tweets as a Case StudyarXiv:2405.11668Hadeel Saadany, Ashraf Tantawy, Constantin Orasan2024 · 3 citations
  92. Evaluating large language models for the tasks of PoS tagging within the Universal Dependency frameworkMateus Machado, Evandro Ruiz2024 · 3 citations
  93. PCR4ALL: A Comprehensive Evaluation Benchmark for Pronoun Coreference Resolution in EnglishXinran Zhao, Hongming Zhang, Yangqiu Song2022 · 3 citations
  94. BPoMP: The Benchmark of Poetic Minimal Pairs – Limericks, Rhyme, and Narrative CoherenceRecent Advances in Natural Language Processing (RANLP)Almas Abdibayev, Allen Riddell, Daniel Rockmore2021 · 3 citations
  95. SiPOS: A Benchmark Dataset for Sindhi Part-of-Speech TaggingRecent Advances in Natural Language Processing (RANLP)Wazir Ali, Zenglin Xu, Jay Kumar2021 · 3 citations
  96. English Out-of-Vocabulary Lexical Evaluation TaskarXiv:1804.04242Han Wang, Ye Wang, Xinxiang Zhang, et al.2018 · 3 citations
  97. Applying the Cognitive Machine Translation Evaluation Approach to ArabicLanguage Resources and Evaluation Conference (LREC)Irina Temnikova, Wajdi Zaghouani, Stephan Vogel, et al.2016 · 3 citations
  98. Concept Based vs. Pseudo Relevance Feedback Performance Evaluation for Information Retrieval SystemarXiv:1403.4362 · International Journal of Computational Linguistics Research, ISSN: 0976-416X, Volume 4, Issue 4, December, 2013, Pages 149-158Mohammed El Amine Abderrahim2013 · 3 citations
  99. Automatic Extraction and Evaluation of Arabic LFG ResourcesLanguage Resources and Evaluation Conference (LREC)Mohammed Attia, Khaled Shaalan, Lamia Tounsi, et al.2012 · 3 citations
  100. A Critical Study of Automatic Evaluation in Sign Language TranslationLanguage Resources and Evaluation Conference (LREC)Shakib Yazdani, Yasser HAMIDULLAH, Cristina España-Bonet, et al.2026 · 2 citations

1 … 5 6 7 8

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/6/