📰 GEM 2026: all accepted papers

76 papers · the most cited first · ← GEM 2026

1. An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability ↗Yusuke Yamauchi, Taro Yano, Masafumi Oyamada3 citations
2. Capturing Epistemic Uncertainty in LLM-Based Soft Labeling ↗Yanru Jiang, Siyu Liang1 citation
3. MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts ↗Jiayi He, Yangmin Huang, Qianyun Du et al.1 citation
4. MetaGraph: A Large-Scale Meta-Analysis of GenAI in Financial NLP (2022–2025) ↗Paolo Pedinotti, Peter Baumann, Nathan Jessurun et al.1 citation
5. Permutation-Consensus Listwise Judging for Robust Factuality Evaluation ↗Tianyi Huang, Nathan Huang, Justin Tang et al.1 citation
6. Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analyses ↗Xanh Ho, Jiahao Huang, Florian Boudin et al.1 citation
7. Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models ↗Priyanka Mary Mammen, Emil Joswin, Shankar Venkitachalam1 citation
8. A Progressive Evaluation Framework for Multicultural Analysis of Story Visualization ↗Janak Kapuriya, Ali Hatami, Paul Buitelaar
9. Are LLM Benchmarks Already Contaminated? A Systematic Review of Contamination Detection Methods ↗Erfan Nourbakhsh, Mohammad Sadegh Sirjani, Amir Mousavi et al.
10. C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning ↗Avni Mittal, Rauno Arike
11. CoSy: Conversational Synthesis for Grounded Question Answering ↗Patrick Huber, Arash Einolghozati, Rylan Conway et al.
12. Complex-IF and Beyond: Expert Rubrics for RLVR ↗Sushant Mehta, Liudas Panavas, Eleanor Fleming et al.
13. Component Transfer Can Exceed Full Model Performance: Investigating Post-Trained Mixture-of-Experts ↗Rabin Tiwari
14. Concord: An Agreement-Aware Multi-Adjudication Pipeline for LLM Evaluation ↗Tyler Bliss, Mahit Verma, Aila Iyer-Singh et al.
15. Cross-Domain Semantic Fidelity Evaluation for Meaning-to-Text Generation ↗Davan Harrison, Marilyn Walker
16. Do Nugget-Based Evaluation Patterns Generalize to List-QA? ↗MohammadJavad Ardestani, Ehsan Kamalloo, Davood Rafiei
17. E-star 12B: Reliable Rubric-Following and Domain-Adaptive SLM Evaluator for Korean Industrial Settings ↗Yonghoon Kwon, Heondeuk Lee, Barom Kang
18. Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate ↗Ali Keramati, Justin Cheok, Jacob Horne et al.
19. EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environments ↗Zefang Liu, Yinzhu Quan
20. Evaluating ASR Quality at Scale on TV Entertainment Platforms ↗Adeep Hande, Kishorekumar Sundararajan, Yidnekachew Endale et al.
21. Evaluating Counterfactual Strategic Reasoning in Large Language Models ↗Dimitrios Georgousis, Maria Lymperaiou, Angeliki Dimitriou et al.
22. Evaluating Multilingual Sentiment Classifiers Using an LLM-Annotated Wikipedia Benchmark ↗Milena Stróżyna, Włodzimierz Lewoniewski, Izabela Czumałowska
23. Evaluating the Reliability of LLMs in Faithfully Updating Text: An Empirical Study ↗Ayan Datta, Paheli Bhattacharya, Rishabh Gupta
24. Exploring Coherence of LLMs in Multilingual Question Answering ↗Stefano Campese, Ivano Lauriola
25. Fine-Tuning vs. RAG for Multi-Hop Question Answering with Novel Knowledge ↗Zhuoyi Yang, Yurun Song, Kyler G. Harris et al.
26. ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual–Language Models through Procedural Plans ↗Ananya Sadana, Yash Kumar Lal, Jiawei Zhou
27. Identifying Where Large Language Models Struggle in Answering Complex Questions ↗Xanh Ho, Florian Boudin, Saku Sugawara et al.
28. IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding ↗Sankalp Jajee, Ashutosh Kumar, Nikunj Kotecha et al.
29. Is GraphRAG Needed? From Basic RAG to Graph-/Agentic Solutions with Context Optimization ↗Long Chen, Ryan Razkenari, Yuxuan Zhou et al.
30. Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunities ↗Nikita Tatarinov, Siddhant Sukhani, Agam Shah et al.
31. Lost in Space: Finding the Right Tokens for Structured Output ↗Sil Hamilton, David Mimno
32. MCJudgeBench: A Benchmark for Constraint-Level Judge Evaluation in Multi-Constraint Instruction Following ↗Jaeyun Lee, Junyoung Koh, Zeynel Tok et al.
33. MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models ↗Weixin Liu, Congning Ni, Shelagh A. Mulvaney et al.
34. Mapping Out the NLP Evaluation Landscape with a Standard Taxonomy of Quality Criteria ↗Anya Belz, Simon Mille, Craig Thomson
35. MedAct: Removing the Human Bottleneck in Benchmarking Clinical LLM Safety ↗Arjun Krishna, Brian Pridgen, Max Silverstein
36. Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual ResultsJan-Thorsten Peter, David Vilar, Tobias Domhan et al.our summary
37. More Yap Less Meaning: Uncovering Self-Improvement Behavior in SLMs ↗Marina Igitkhanian, Erik Arakelyan
38. NanoFlux: Adversarial Dual-LLM Evaluation and Distillation for Multi-Domain Reasoning ↗Raviteja Anantha, Soheil Hor, Teodor Nicola Antoniu et al.
39. Near-Miss: Latent Policy Failure Detection in Agentic Workflows ↗Ella Rabinovich, David Boaz, Naama Zwerdling et al.
40. Not All Tokens Are Equal: Per-Dimension Top-K Pooling for Adversarially Robust BERT Classification ↗Manoranjan Dash, Shivam Anand Aralikatti, Shanay Sheth et al.
41. Position: A Semiotic-Hermeneutic Approach to Evaluating Meaning in LLM Summaries via the Inductive Conceptual Rating Metric ↗Natalie Perez, Sreyoshi Bhaduri, Aman Chadha
42. Position: Evaluation Scores Are Perishable Knowledge Claims ↗Sankalp Gilda, Shlok Gilda
43. Position: Scores Without Context? Rethinking the Role of Evaluation in the Era of LLMs ↗Jiawei Zhou
44. Position: Toward a Metric Typology for Language Model Evaluation ↗Jasper Kyle Catapang
45. Position: What Are We Measuring? Rethinking Evaluation in Natural Language Generation ↗Wajdi Zaghouani
46. Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations ↗Sachin Kumar
47. Process Standardisation for Human Evaluation of NLP System Outputs ↗Craig Thomson, Javier González Corbelle, Anya Belz
48. RBCorr: Response Bias Correction in Language Models ↗Om Bhatt, Anna A Ivanova
49. RE-AD: Real-Time Requirement Adherence for Data Labeling ↗Siddarth Malreddy, Ishan Nigam, Akshay Arora et al.
50. Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests ↗Manar Ali, Judith Sieker, Sina Zarrieß et al.
51. Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents ↗Anh Ta, Junjie Zhu, Shahin Shayandeh
52. ReproHum \#0031–01: Reproducing a Human Readability Evaluation for Question–Answer Generation Systems ↗Manuela Hürlimann, Mark Cieliebak
53. ReproHum \#0033-05: Human Evaluation Report on “Generating Scientific Definitions with Controllable Complexity” ↗Ines Arous, Jackie Chi Kit Cheung
54. ReproHum \#0124-03: Reproducing Human Scores on Neural REG Models ↗Maurice Langner
55. ReproHum \#0669-08: Reproducing a Recipe for Arbitrary Text Style Transfer with LLMsSaad Mahamoodour summary
56. ReproHum \#0866-04: Variability in Human Judgments of Sociopolitical Acceptability Across Studies ↗Rui Fan, Guanyi Chen
57. ReproNLP 2026: A Third Replication of the Human Evaluation of a QAG System for Children's Storybooks ↗Marcel Mroczek, Chiara Albarello, Paul-Emmanuel Floch et al.
58. Response Content Units: Evaluating Completeness and Proactiveness in Medical Open-Response Question Answering ↗Yongsin Park, Wen-wai Yim, Emma McKibbin et al.
59. SAUCE: Summary Analysis Using Conversation Entailment ↗Man-Ling Sung, Hemanth Kandula, Jeff Ma et al.
60. Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence ↗Harshavardhan
61. Semantic vs. Structural Signals: Log-Probability and LLM-as-a-Judge for Reference-Free Code Evaluation ↗Dmitriy Fedrushkov, Yulong He, Ivan Smirnov et al.
62. Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks ↗Aditi Gupta, Neel Mishra, Kushagra Trivedi et al.
63. Stability vs. Manipulability: Evaluating Robustness Under Post-Decision Interaction in LLM Judges ↗Srimonti Dutta, Akshata Kishore Moharir
64. Sycophancy Negatively Affects LLM-as-a-Judge in Conflict Evaluation ↗Naghmeh Farzi, Laura Dietz, Samuel Carton
65. TSAQA: Time Series Analysis Question And Answering Benchmark ↗Baoyu Jing, Sanhorn Chen, Lecheng Zheng et al.
66. Teaching Values to Machines: Simulating Human-Like Behavior in LLMs ↗Asaf Yehudai, Naama Rozen, Ariel Gera
67. Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models ↗Zefang Liu, Nam H Nguyen, Yinzhu Quan et al.
68. Text Analytics Evaluation Framework: A Case Study on LLMs and Social Media ↗Yuefeng Shi, Nedjma Ousidhoum, Jose Camacho-Collados
69. The Shared Task on Reproducibility of Evaluations in NLP (ReproNLP) 2026: Overview and Results ↗Anya Belz, Craig Thomson, Javier González Corbelle
70. The Silent Vote: Improving Zero-Shot LLM Reliability by Aggregating Semantic Neighborhoods ↗Sanket Badhe, Priyanka Tiwari, Deep Shah
71. Token Cost Inequality: Measuring Tokenization Disparities Across Scripts in Roman Urdu and Urdu ↗Waleed Jamil, Saima Rafi, Yanchao Yu
72. Tool-Aware Planning for Contact-Center Analytics: Evaluating LLMs through Lineage-Guided Query Decomposition ↗Varun Nathan, Shreyas Guha, Ayush Kumar
73. VAIDYA: Validated Agents for Intelligent Diagnosis and Yielded Analysis ↗Kalash Shah, Gautam Bhutani, Rohitaswa Sarbhangia et al.
74. When Users Are Happy but Agents Are Wrong: Multi-Dimensional Evaluation of Tool-Augmented Dialogue ↗Tanya Shourya, Yingfan Wang, Zhaoyi Joey Hou et al.
75. WildIFEval: Instruction Following in the Wild ↗Gili Lior, Asaf Yehudai, Ariel Gera et al.
76. “Be My Cheese?”: Cultural Nuance Benchmarking for Machine Translation in Multilingual LLMs ↗Madison Van Doren, Casey Ford, Jennifer Barajas et al.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/gem/2026/papers/