Elamur Research⌕
☰
home / topics / Evaluation & benchmarks / page 8

📏 Evaluation & benchmarks

684 papers · page 8 of 8 · 626–684 · the most cited first · all topics

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

 Papers (684)

1 … 7 8

  1. HotelCheckSpan: A Benchmark Dataset for LLM FaithfulnessLanguage Resources and Evaluation Conference (LREC)Patricia Schmidtova, Ondrej Dusek, Saad Mahamood2026
  2. HumaniCA: A Benchmark Resource for the Detection of Users' Ascription of Humanness to Conversational AgentsLanguage Resources and Evaluation Conference (LREC)Sabrina Villata, Amon Rapp, Luigi Di Caro, et al.2026
  3. HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship DetectionLanguage Resources and Evaluation Conference (LREC)Luke S. Patterson, Li Wang, Adam Faulkner2026
  4. InstructSum: A Benchmark to Evaluate Instruction-Following Capability of Large Language Models in SummarizationLanguage Resources and Evaluation Conference (LREC)Kosuke Nishida, Kyosuke Nishida, Itsumi Saito2026
  5. J-ClinicalBench: A Benchmark for Evaluating Large Language Models on Practical Clinical Tasks in JapaneseLanguage Resources and Evaluation Conference (LREC)Seiji Shimizu, Tomohiro Nishiyama, HISADA Shohei, et al.2026
  6. KinyCOMET: Automatic Evaluation of Machine Translation Systems for Kinyarwanda–EnglishLanguage Resources and Evaluation Conference (LREC)Prince Chris Mazimpaka, Jan Nehring, Samuel Rutunda, et al.2026
  7. Logic-Level Evaluation of Logical Table-to-Text GenerationConference on Computational Natural Language Learning (CoNLL)Lena Trigg, Dean F. Hougen, Ahsan Bilal2026
  8. Lost in Translation: Repurposing semantic similarity benchmarks for evaluating lexical-semantic consistency in LLM-based machine translationWorkshop on Resources and Representations for Under-Resourced Languages and DomainsQuin Ye, Jelke Bloem2026
  9. Machine translation Evaluation Eng-Thai MQM Ranking datasetConference of the European Chapter of the Association for Computational Linguistics (EACL)Phichet Phuangrot, Natdanai Trintawat, Kanawat Vilasri, et al.2026
  10. Meaning Over Morphology: A Multi-Metric Benchmark of LLMs for Bangla Dialect TranslationProceedings of the First Workshop on Dialects in NLP –- A Resource PerspectiveSoumik Deb Niloy, Subhey Sadi Rahman, Mahbub E Sobhani, et al.2026
  11. Meaning-Making Process and Error Dynamics in ChatGPT-Mediated TranslationAnnual Conference of the European Association for Machine Translation (EAMT)Iulia Mihalache, María-José Varela Salinas2026
  12. Metric Grammars: A usage-based grammatical formalism that supports generation, parsing and morphological innovationConference on Computational Natural Language Learning (CoNLL)Whitney Tabor, Hyosun Lee2026
  13. MEUR: A Benchmark for Evaluating Vision-Language Models on Multimodal Event Understanding and ReasoningLanguage Resources and Evaluation Conference (LREC)Zimu Wang, Yuqi Wang, Tong Chen, et al.2026
  14. MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine TranslationAnnual Meeting of the Association for Computational Linguistics (ACL)Parker Riley, Daniel Deutsch, Mara Finkelstein, et al.2026
  15. mSCoRe: A Multilingual and Scalable Benchmark for Skill-based Commonsense ReasoningLanguage Resources and Evaluation Conference (LREC)Nghia Trung Ngo, Franck Dernoncourt, Thien Huu Nguyen2026
  16. MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn DialoguesFindings of the Association for Computational LinguisticsYaning Pan, Qianqian Xie, Guohui Zhang, et al.2026
  17. Multi-dimensional Evaluation of Character-Authentic Dialogue Models Learned from Question-Answer DataLanguage Resources and Evaluation Conference (LREC)Atsushi Otsuka, Kazuya Matsuo, Kenta Hama, et al.2026
  18. NegNLI-BR: A Brazilian Portuguese Benchmark for Negation in Natural Language InferenceLanguage Resources and Evaluation Conference (LREC)Matheus Westhelle, Viviane Moreira2026
  19. On Temperature-Constrained Non-Deterministic Machine Translation: Potential and EvaluationFindings of the Association for Computational LinguisticsWeichuan Wang, Mingyang Liu, Chen Ma, et al.2026
  20. OpenSubtitles2024: A Massively Parallel Dataset of Movie Subtitles for MT Development and EvaluationLanguage Resources and Evaluation Conference (LREC)Joerg Tiedemann, Hengyu Luo2026
  21. Parsing Arabic Dialects Revisited: New Benchmarks, Models, and InsightsAhmed Farouk Zakaria Elshabrawy, Go Inoue, Muhammed AbuOdeh, et al.2026
  22. Persona-Aware Evaluation of Cognitive Bias in LLMs: From Benchmark to Applied Decision-MakingLanguage Resources and Evaluation Conference (LREC)Katsumasa Yoshikawa, Junya Takayama, Takato Yamazaki2026
  23. Quantifying Metric and Model Agreement in Bias Evaluation of Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Arash Asgari, Huan Wu, Amirreza Naziri, et al.2026
  24. Quantifying the Impact of Translation Errors on Multilingual LLM EvaluationAnnual Meeting of the Association for Computational Linguistics (ACL)Klaudia Thellmann, Bernhard Stadler, Michael Färber, et al.2026
  25. Questionnaire Meets LLM: A Benchmark and Empirical Study of Structural Skills for Understanding Questions and ResponsesLanguage Resources and Evaluation Conference (LREC)Duc-Hai Nguyen, Vijayakumar Nanjappan, Barry O'Sullivan, et al.2026
  26. QuranicMMLU: A Cognitively-Aware Benchmark for Evaluating Generative AI Solutions on Quranic Linguistic KnowledgearXiv:2609.22038 · Arabic NLP 2026Rawan El Ghali; Umm Kulsoom; Anas Madkoor; Dima Faris Alsaudi; Roaa Abdelmagid; Roaa Ibrahim; Raghad Mousa; Hamza Aljaji; Abdullah Khanafer; Abdallah Alkanani; Salah Feras Alali; Rawan Khaled Mohamed;, Ehsaneddin Asgari2026
  27. Reference-free Evaluation at Inference for NER/NEL over OCRed Historical TextsLanguage Resources and Evaluation Conference (LREC)Tien-Nam Nguyen, Adam Jatowt, Ahmed Hamdi, et al.2026
  28. Rethinking Evaluation in Retrieval-Augmented Personalized Dialogue: A Cognitive and Linguistic PerspectiveLanguage Resources and Evaluation Conference (LREC)Tianyi Zhang, David Traum2026
  29. Rethinking Gender Annotation for Bias Evaluation in Machine Translation: Can LLMs Improve Reliability?Workshop on Gender-Inclusive Translation TechnologiesChiara Manna, Argentina Anna Rescigno, Eva Vanmassenhove2026
  30. Rethinking Role-Playing Evaluation: Anonymous Benchmarking and A Systematic Study of Personality EffectsAnnual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)Ji-Lun Peng, Yun-Nung Chen2026
  31. SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV SeriesAnnual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)Galann Pennec, Zhengyuan Liu, Nicholas Asher, et al.2026
  32. Scoring the Translation: On Target Automatic Keyword-Based Evaluation of Machine Translation in the Sports DomainLanguage Resources and Evaluation Conference (LREC)Steinþór Steingrímsson, Einar Freyr Sigurðsson2026
  33. Semantic Parsing for Evaluating Large Language Models: Separating Linguistic Abilities with YARNLanguage Resources and Evaluation Conference (LREC)Rémi DE VERGNETTE, Maxime Amblard2026
  34. STRUDEL: Unrolling a Benchmark for Evaluating Vision-Language Models on Structured Diagram Understanding across DomainsLanguage Resources and Evaluation Conference (LREC)Daniel Steinigen, Lucie Flek, Sebastian Houben2026
  35. The Potential for Misleading Results in Text Sanitisation with Standard Evaluation MetricsLanguage Resources and Evaluation Conference (LREC)Dan Zhang, Mark Anderson2026
  36. To Overfit or Not to Overfit? An Evaluation of HTR Workflow on 17Th-18Th Century French CorpusLanguage Resources and Evaluation Conference (LREC)Marine Tiger2026
  37. TryggLLM: A Benchmark for Evaluating LLM Safety in NorwegianLanguage Resources and Evaluation Conference (LREC)Samia Touileb, Truls Pedersen, Isabell Stinessen Haugen2026
  38. ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset & BenchmarkLanguage Resources and Evaluation Conference (LREC)Tung X. Nguyen, Nhu Vo, Giang-Son Nguyen, et al.2026
  39. Which Way Does Time Flow? A Psychophysics-Grounded Evaluation for Vision–Language ModelsLanguage Resources and Evaluation Conference (LREC)Shiho Matta, Lis Kanashiro Pereira, Peitao Han, et al.2026
  40. Who Generates More Empathetic Responses–-Humans or LLMs? A Comparative Evaluation with Human and LLM JudgesConference on Computational Natural Language Learning (CoNLL)Anuradha Welivita, Fawzia Zeitoun, Pearl Pu2026
  41. YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language ModelsMahmoud Reda, Salam Khalifa, Reham Marzouk, et al.2026
  42. ALHD: A Large-Scale and Multigenre Benchmark Dataset for Arabic LLM-Generated Text DetectionarXiv:2510.03502Ali Khairallah, Arkaitz Zubiaga2025
  43. ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection BenchmarkarXiv:2509.22808Mohamed Elsetohy, Alhassan Ehab, Ali Mekky, et al.2025
  44. Assessing the Agreement Competence of Large Language ModelsAlba Táboas García, Leo Wanner2025
  45. Evaluating the Performance of Transformers in Translating Low-Resource Languages through AkkadianProceedings of the First Workshop on Comparative Performance Evaluation: From Rules to Language ModelsDaniel A. Jones, Ruslan Mitkov2025
  46. Evaluation on Chinese Multiturn Instruction Following Capability of Large Language Models (大语言模型的中文多轮指令遵循能力评测研究)Journal of Chinese Information ProcessingQin Zhu (朱秦), Junliang He (何俊亮), Xipeng Qiu (邱锡鹏), et al.2025
  47. Proceedings of the 1st Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian Languages (BHASHA 2025)Workshop on Benchmarks, Harmonization, Annotation, and Standardization for Human-Centric AI in Indian LanguagesArnab Bhattacharya, Pawan Goyal, Saptarshi Ghosh, et al.2025
  48. Robustness Evaluation of ChatGPT Against Chinese Adversarial Attacks (中文对抗攻击下的ChatGPT鲁棒性评估)Journal of SoftwareYunting Zhang (张云婷), Lin Ye (叶麟), Baisong Li (李柏松), et al.2025
  49. Towards stable AI systems for Evaluating Arabic PronunciationsarXiv:2508.19587 · 4th International Conference on NLP and Machine Learning Trends 2025Hadi Zaatiti, Hatem Hajri, Osama Abdullah, et al.2025
  50. Building and evaluating a Japanese-English bilingual model by vocabulary-replacing continual pretraining (語彙置換継続事前学習による日英バイリンガルモデルの構築と評価)Annual Meeting of the Association for Natural Language Processing麻場直喜, 野崎雄太, 中島大, et al.2024
  51. Evaluating Large Language Models: A Survey of Research Progress (大语言模型评测综述)Journal of Chinese Information ProcessingWen Luo (罗文), Houfeng Wang (王厚峰)2024
  52. Evaluating the Effectiveness of the Foundational Models for Q&A Classification in Mental Health carearXiv:2406.15966Hassan Alhuzali, Ashwag Alasmari2024
  53. Generation and Evaluation of English Grammar Multiple-Choice Cloze ExercisesNicolò Donati, Matteo Periani, Paolo Di Natale, et al.2024
  54. Construction of Chinese Sentence-Level Gender-Unbiased Data Set and Evaluation of Gender Bias in Pre-Training Language Model (中文句子级性别无偏数据集构建及预训练语言模型的性别偏度评估)Journal of Chinese Information ProcessingJishun Zhao (赵继舜), Bingjie Du (杜冰洁), Pengyuan Liu (刘鹏远), et al.2023
  55. The Truth, The Whole Truth, and Nothing but the Truth: A New Benchmark Dataset for Hebrew Text Credibility AssessmentBen Hagag, Reut Tsarfaty2023
  56. Qu'apporte BERT à l'analyse syntaxique en constituants discontinus ? Une suite de tests pour évaluer les prédictions de structures syntaxiques discontinues en anglais (What does BERT contribute to discontinuous constituency parsing ? A test suite to evaluate discontinuous constituency structure predictions in English)JEP-TALN-RECITAL (the French-speaking conferences on speech and natural language processing)Maximin Coavoux2020
  57. Evaluating Lemmatization Models for Machine-Assisted Corpus-Dictionary LinkageKevin Black, Eric Ringger, Paul Felt, et al.2014
  58. Towards a Semantic Annotation of English Television News - Building and Evaluating a Constraint Grammar FrameNetPacific Asia Conference on Language, Information and Computation (PACLIC)Shaohua Yang, Hai Zhao, Bao-liang Lu2012
  59. Collection and Evaluation of Broadcast News Data for ArabicLanguage Resources and Evaluation Conference (LREC)Mohamed Afify, Ossama Emam2004

1 … 7 8

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/