Elamur Research⌕
☰
home / papers / 2026 / MÖVE: A Holistic LLM Benchmark for the German Public Sector

MÖVE: A Holistic LLM Benchmark for the German Public Sector

Paper · 2026 · Camilla Dalerci, Thilo Michael, Robin Schaefer et al.

Summary

Public administrations in Germany are adopting LLMs, but they usually pick models by hype or by where the vendor is based. Existing benchmarks do not help much: most are in English, built around US content, and measure only task quality. MÖVE (Modelle für die Öffentliche Verwaltung Evaluieren) is a German-language benchmark for this setting. It tests 39 models on ten datasets, some of which are in-house gold- and silver-standard sets drawn from administrative domains.

The benchmark has two axes. The performance axis covers summarization, question answering and topic extraction. The governance axis covers hallucination, energy use, provider transparency, and alignment with constitutional values and with German party positions. Scoring mixes classical NLP metrics, embedding similarity and LLM-as-a-judge.

No model wins everywhere. Leaders change from task to task, and parameter count says little about quality. The authors also audit the benchmark itself: its statistical precision, judge reliability, the effect of the private data on rankings, prompt sensitivity, and how sound the energy estimates are. MÖVE gives procurement a documented, multi-criteria basis and is maintained as a continuously updated public leaderboard.

Key points

  • The benchmark covers 39 models, ten German datasets and three performance tasks, all with German prompts. The governance side includes a Transparency Matrix questionnaire about providers.
  • The performance runs took 601,618 queries and an estimated 1,564 kWh. On the Swedish grid this is about 75 kgCO2eq; the same workload on German infrastructure would emit noticeably more.
  • Energy use is highly concentrated. DeepSeek R1, Mistral-Large-3 and GPT-4o account for 48.0% of the total, and the top ten models for 71.2%.
  • The hallucination score is currently built only from faithfulness scores on the QA task.
  • A single, fully documented LLM judge is used to keep costs down, even though the paper's own analysis finds systematic scoring differences between judges. Some models wrote summaries more than three times the reference length without being explicitly penalized.

Interesting ideas

  • I1. The benchmark is treated as an object of study: the authors measure ranking precision (confidence intervals), judge stability, prompt sensitivity and the validity of their energy proxies, rather than just reporting scores.
  • I2. Judges are checked twice: once by re-scoring with the same judge (within-judge stability) and once by comparing different judges (inter-judge agreement).
  • I3. Private in-house gold and silver sets act as a guard against contamination, and the authors measure how much these sets shift the rankings.
  • I4. Hallucination is defined as unfaithfulness to the provided context. The planned extension tests what happens when the context is missing or self-contradictory and counts how often the model abstains.
  • I5. The paper notes that content metrics miss violations of format and length, and proposes a dedicated instruction-following score.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): every evaluation report could include confidence intervals and a prompt/format-perturbation check, so that a change in score can be told apart from noise.
  • I2 (learn): LLM teacher tiers could be vetted the same way, by re-asking each teacher and cross-checking teachers before their labels enter the silver data.
  • I3 (answer): keep a held-out set of tests and word problems written in-house and report how scores on it diverge from scores on public sets.
  • I4 (answer): readers could be scored on whether they decline to answer when the text world lacks the fact or contains conflicting states, instead of guessing.
  • I5 (back): generated text could be checked against explicit constraints such as length relative to the source and required form, alongside meaning-preservation scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BundesdruckereiBerlin · Germany2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Camilla Dalerci, Thilo Michael, Robin Schaefer and Daniel Weinland. 2026. MÖVE: A Holistic LLM Benchmark for the German Public Sector. arXiv:2606.13111.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2606.13111 · arXiv 2606.13111

@misc{dalerci2026move,
  title = {{MÖVE: A Holistic LLM Benchmark for the German Public Sector}},
  author = {Camilla Dalerci, Thilo Michael, Robin Schaefer and Daniel Weinland},
  year = {2026},
  doi = {10.48550/arxiv.2606.13111},
  eprint = {2606.13111},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.13111}
}

References

135 references, 57 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jan Etscheid, Jörn von Lucke, and Felix Stroh. Künstliche intelligenz in der öffentlichen verwaltung. Digitalakademie@ BW & Fraunhofer IAO, Stuttgart, pages 11–12, 2020. find
  2. Statistisches Bundesamt (Destatis). Öffentlicher Dienst 2024: Mehr Beschäftigte für Bildung und Kinderbetreuung. https://www.destatis.de/DE/Presse/ Pressemitteilungen/2025/06/PD25_212_741.html, 2025. Pressemitteilung Nr. 212 vom 23. Juni 2025. Accessed: 2026-05-07. ↗
  3. PwC Deutschland. Wie öffentliche Institutionen ihre Beschäftigten strategisch binden sollten. https://blogs.pwc.de/de/oeffentlicher-sektor-zukunft-gestalten/article/252701/ wie-oeffentliche-institutionen-ihre-beschaeftigten-strategisch-binden-sollten/, January 2026. Blog post, Öffentlicher Sektor – Zukunft gestalten. Accessed: 2026-05-07. ↗
  4. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding, January 2021. URL http://arxiv.org/abs/2009.03300. arXiv:2009.03300 [cs]. ↗
  5. Aarohi Srivastava et al. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models, June 2023. URL http://arxiv.org/ abs/2206.04615. arXiv:2206.04615 [cs]. ↗
  6. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Tal Linzen, Grzegorz Chrupała, and Afra Alishahi, editors, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Brussels, Belgium, November 2018. Association for Computational Linguistics. doi: 10.18653/v1/W18-5446. URL https://aclanthology.org/W18-5446/. ↗
  7. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. SuperGLUE: a stickier benchmark for general-purpose language understanding systems. Curran Associates Inc., Red Hook, NY, USA, 2019.
  8. Shivalika Singh et al. Global MMLU: Understanding and addressing cultural and linguistic biases in multilingual evaluation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18761–18799, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025. acl-long.919. URL https://aclanthology.org/2025.acl-long.919/. ↗
  9. Klaudia Thellmann, Bernhard Stadler, Michael Fromm, Jasper Schulze Buschhoff, Alex Jude, Fabio Barth, Johannes Leveling, Nicolas Flores-Herr, Joachim Köhler, René Jäkel, and Mehdi Ali. Towards multilingual llm evaluation for european languages, 2024. URL https://arxiv.org/abs/2410.08928. preprint. ↗
  10. Percy Liang et al. Holistic Evaluation of Language Models, October 2023. URL http://arxiv.org/abs/2211.09110. arXiv:2211.09110 [cs]. ↗
  11. Oscar Sainz, Jon Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776–10787, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-emnlp.722. URL https://aclanthology.org/2023.findings-emnlp.722/. ↗
  12. Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 12039–12050, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-acl.716. URL https://aclanthology.org/2024.findings-acl.716/. ↗
  13. Joshua Harris, Fan Grayson, Felix Feldman, Timothy Laurence, Toby Nonnenmacher, Oliver Higgins, Leo Loman, Selina Patel, Thomas Finnie, Samuel Collins, and Michael Borowitz. Healthy llms? benchmarking llm knowledge of uk government public health information, 2025. URL https://arxiv.org/abs/ 2505.06046. preprint. ↗
  14. Neil Majithia, Rajat Shinde, Zo Chapman, Prajun Trital, Jordan Decker, Manil Maskey, Elena Simperl, and Nigel Shadbolt. The citizenquery benchmark: A novel dataset and evaluation pipeline for measuring llm performance in citizen query tasks, 2026. URL https://arxiv.org/abs/2602.04064. ↗
  15. Shuo Liu, Lin Zhang, Weidong Liu, Jianfeng Zhang, Donghui Gao, and Xiaofeng Jia. The evaluation framework and benchmark for large language models in the government affairs domain. ACM Trans. Intell. Syst. Technol., 16(6), November 2025. ISSN 2157-6904. doi: 10.1145/3716854. URL https://doi.org/10.1145/ 3716854. ↗
  16. Jonathan Rystrøm, Chris Schmitz, Karolina Korgul, Jan Batzner, and Chris Russell. Agent benchmarks fail public sector requirements, 2026. URL https://arxiv.org/abs/2601.20617. ↗
  17. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (AI Act), 2024. URL https://eur-lex.europa.eu/eli/reg/2024/1689/oj. ↗
  18. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
  19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. URL http://arxiv.org/abs/1810.04805. ↗
  20. Tom B. Brown et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
  21. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 93–104, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1009. URL https://aclanthology.org/ D18-1009/. ↗
  22. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1472. URL https://aclanthology.org/P19-1472/. ↗
  23. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: an adversarial winograd schema challenge at scale. Commun. ACM, 64(9):99–106, August 2021. ISSN 0001-0782. doi: 10.1145/3474381. URL https://doi.org/10.1145/3474381. ↗
  24. Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, and Zhenzhong Lan. CLUE: A Chinese language understanding evaluation benchmark. In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Linguistics, pages 4762–4772, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. doi: 10.18653/v1/2020.coling-main.419. URL https://aclanthology.org/2020.coling-main.419/. ↗
  25. Yubo Wang et al. Mmlu-pro: a more robust and challenging multi-task language understanding benchmark. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9798331314385.
  26. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=Ti67584b98. ↗
  27. Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. Healthbench: Evaluating large language models towards improved human health, 2025. URL https://arxiv.org/abs/2505.08775. ↗
  28. Jakob Hauser, Daniel Kondor, Jenny Reddish, Majid Benam, Enrico Cioni, Federica Villa, James S. Bennett, Daniel Hoyer, Pieter Francois, Peter Turchin, and R. Maria del Rio-Chanona. Large language models’ expert-level global history knowledge benchmark (hist-llm). In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN 9798331314385. find
  29. Milena de Swart, Floris Den Hengst, and Jieying Chen. Detecting linguistic bias in government documents using large language models. In Proceedings of the ACM on Web Conference 2025, WWW ’25, page 5034–5044, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400712746. doi: 10.1145/3696410.3714526. URL https://doi.org/10.1145/3696410.3714526. ↗
  30. David Ifeoluwa Adelani et al. IrokoBench: A new benchmark for African languages in the age of large language models. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2732–2757, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long.139. URL https://aclanthology.org/2025.naacl-long.139/. ↗
  31. Yosephine Susanto, Adithya Venkatadri Hulagadri, Jann Railey Montalan, Jian Gang Ngui, Xianbin Yong, Wei Qi Leong, Hamsawardhini Rengarajan, Peerat Limkonchotiwat, Yifan Mai, and William Chandra Tjhi. SEA-HELM: Southeast Asian holistic evaluation of language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, pages 12308–12336, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl.636. URL https://aclanthology.org/2025.findings-acl.636/. ↗
  32. Ziqi Yin, Hao Wang, Kaito Horio, Daisuke Kawahara, and Satoshi Sekine. Should we respect LLMs? a cross-lingual study on the influence of prompt politeness on LLM performance. In James Hale, Kushal Chawla, and Muskan Garg, editors, Proceedings of the Second Workshop on Social Influence in Conversations (SICon 2024), pages 9–35, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.sicon-1.2. URL https://aclanthology.org/2024.sicon-1.2/. ↗
  33. Arda Yüksel, Abdullatif Köksal, Lütfi Kerem Senel, Anna Korhonen, and Hinrich Schuetze. TurkishMMLU: Measuring massive multitask language understanding in Turkish. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7035–7055, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-emnlp.413. URL https://aclanthology.org/2024.findings-emnlp.413/. ↗
  34. Guijin Son, Hanwool Lee, Sungdong Kim, Seungone Kim, Niklas Muennighoff, Taekyoon Choi, Cheonbok Park, Kang Min Yoo, and Stella Biderman. KMMLU: Measuring massive multitask language understanding in Korean. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4076–4104, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long. 206. URL https://aclanthology.org/2025.naacl-long.206/. ↗
  35. François Chollet. On the measure of intelligence, 2019. URL https://arxiv.org/ abs/1911.01547. ↗
  36. Francois Chollet, Mike Knoop, Gregory Kamradt, Bryan Landers, and Henry Pinkard. Arc-agi-2: A new challenge for frontier ai reasoning systems, 2025. URL https://arxiv.org/abs/2505.11831. ↗
  37. Ine Gevers, Victor De Marez, Jens Van Nooten, Jens Lemmens, Andriy Kosar, Ehsan Lotfi, Nikolay Banar, Pieter Fivez, Luna De Bruyne, and Walter Daelemans. In benchmarks we trust ... or not? In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23662–23676, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main.1208. URL https://aclanthology.org/2025.emnlp-main.1208/. ↗
  38. Deborah Raji, Emily Denton, Emily M. Bender, Alex Hanna, and Amandalynne Paullada. Ai and the everything in the whole wide world benchmark. In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021. ↗
  39. Andrew M. Bean et al. Measuring what matters: Construct validity in large language model benchmarks. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. URL https://openreview.net/forum?id=mdA5lVvNcU. ↗
  40. Samuel R. Bowman and George Dahl. What will it take to fix benchmarking in natural language understanding? In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4843–4855, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021. naacl-main.385. URL https://aclanthology.org/2021.naacl-main.385/. ↗
  41. Yucheng Li, Yunhao Guo, Frank Guerin, and Chenghua Lin. An open-source data contamination report for large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 528–541, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024.findings-emnlp.30. URL https://aclanthology.org/2024.findings-emnlp. 30/. ↗
  42. Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. Proving test set contamination in black-box language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=KS8mIvetg2. ↗
  43. Zerui Cheng et al. Benchmarking is broken – don’t let ai be its own judge, 2025. URL https://arxiv.org/abs/2510.07575. ↗
  44. Anka Reuel, Amelia Hardy, Chandler Smith, Max Lamparth, Malcolm Hardy, and Mykel Kochenderfer. Betterbench: Assessing AI benchmarks, uncovering issues, and establishing best practices. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=hcOq2buakM. ↗
  45. Julian Kupiec, Jan Pedersen, and Francine Chen. A trainable document summarizer. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, pages 68–73, 1995. find
  46. Yihong Gong and Xin Liu. Generic text summarization using relevance measure and latent semantic analysis. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’01, pages 19–25, New York, NY, USA, September 2001. Association for Computing Machinery. ISBN 978-1-58113-331-8. doi: 10.1145/383952.383955. URL https://doi.org/10.1145/383952.383955. ↗
  47. Jianpeng Cheng and Mirella Lapata. Neural Summarization by Extracting Sentences and Words, July 2016. URL http://arxiv.org/abs/1603.07252. arXiv:1603.07252 [cs]. ↗
  48. Kevin Knight and Daniel Marcu. Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence, 139(1): 91–107, 2002. ↗
  49. Alexander M. Rush, Sumit Chopra, and Jason Weston. A Neural Attention Model for Abstractive Sentence Summarization, September 2015. URL http://arxiv.org/abs/1509.00685. arXiv:1509.00685 [cs]. ↗
  50. Abigail See, Peter J. Liu, and Christopher D. Manning. Get To The Point: Summarization with Pointer-Generator Networks, April 2017. URL http://arxiv.org/abs/1704.04368. arXiv:1704.04368 [cs]. ↗
  51. Huan Yee Koh, Jiaxin Ju, Ming Liu, and Shirui Pan. An Empirical Survey on Long Document Summarization: Datasets, Models, and Metrics. ACM Computing Surveys, 55(8):1–35, August 2023. ISSN 0360-0300, 1557-7341. doi: 10.1145/3545176. ↗
  52. Yang Zhang, Hanlei Jin, Dan Meng, Jun Wang, and Jinghua Tan. A comprehensive survey on automatic text summarization with exploration of llm-based methods. Neurocomputing, page 131928, 2025. ↗
  53. Ramesh Nallapati, Bowen Zhou, Cicero Dos Santos, Çağlar Gulçehre, and Bing Xiang. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of the 20th SIGNLL conference on computational natural language learning, pages 280–290, 2016.
  54. Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. pages 1797–1807, 2018.
  55. Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. Efficient attentions for long document summarization. pages 1419–1436, 2021.
  56. Anastassia Kornilova and Vladimir Eidelman. BillSum: A corpus for automatic summarization of US legislation. pages 48–56, 2019.
  57. Dennis Aumiller, Ashish Chouhan, and Michael Gertz. Eur-lex-sum: A multi-and cross-lingual dataset for long-form summarization in the legal domain. arXiv preprint arXiv:2210.13448, 2022. ↗
  58. Vishvaksenan Rasiah, Ronja Stern, Veton Matoshi, Matthias Stürmer, Ilias Chalkidis, Daniel E. Ho, and Joel Niklaus. Scale: Scaling up the complexity for advanced language model evaluation, 2023. find
  59. Chin-Yew Lin. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/. ↗
  60. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL ’02, page 311, Philadelphia, Pennsylvania, 2001. Association for Computational Linguistics. doi: 10.3115/1073083.1073135. ↗
  61. Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. Summeval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics, 9: 391–409, 2021. ↗
  62. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. BERTScore: Evaluating Text Generation with BERT, February 2020. URL http://arxiv.org/abs/1904.09675. arXiv:1904.09675 [cs]. ↗
  63. Ansar Aynetdinov and Alan Akbik. SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity, January 2024. URL http://arxiv.org/abs/2401.17072. arXiv:2401.17072 [cs]. ↗
  64. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, December 2023. URL http://arxiv.org/abs/2306.05685. arXiv:2306.05685 [cs]. ↗
  65. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634, 2023. ↗
  66. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Llms-as-judges: a comprehensive survey on llm-based evaluation methods. arXiv preprint arXiv:2412.05579, 2024. ↗
  67. Tanya Goyal, Junyi Jessy Li, and Greg Durrett. News summarization and evaluation in the era of gpt-3. arXiv preprint arXiv:2209.12356, 2022. ↗
  68. Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics, 12:39–57, 2024. ISSN 2307-387X. ↗
  69. Anna Rogers, Matt Gardner, and Isabelle Augenstein. QA dataset explosion: A taxonomy of NLP resources for question answering and reading comprehension. ACM Computing Surveys, 55:1–45, 2023. doi: 10.1145/3560260. ↗
  70. Penghao Zhu, Zhiwei Lin, Zijian Wang, Xiaodan Liang, et al. Retrieval-augmented generation for AI-generated content: A survey. arXiv preprint arXiv:2402.19473, 2024. ↗
  71. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, volume 28, 2015. ↗
  72. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras, editors, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653/v1/ D16-1264. URL https://aclanthology.org/D16-1264/. ↗
  73. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879. Association for Computational Linguistics, 2017.
  74. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086– 6096. Association for Computational Linguistics, 2019.
  75. Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring How Models Mimic Human Falsehoods, May 2022. URL http://arxiv.org/abs/2109. 07958. arXiv:2109.07958 [cs]. ↗
  76. Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789. Association for Computational Linguistics, 2018. — Know What You Don't Know: Unanswerable Questions for SQuAD
  77. Tom Kwiatkowski et al. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453– 466, 2019.
  78. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. pages 1601–1611, 2017.
  79. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. ELI5: Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558– 3567. Association for Computational Linguistics, 2019.
  80. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637. Association for Computational Linguistics, 2020.
  81. Patrick Lewis, Barlas Oguz, Ruben Rinott, Sebastian Riedel, and Holger Schwenk. MLQA: Evaluating cross-lingual extractive question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7315–7330. Association for Computational Linguistics, 2020.
  82. Timo Möller, Julian Risch, and Malte Pietsch. GermanQuAD and GermanDPR: Improving non-English question answering and passage retrieval. In Adam Fisch, Alon Talmor, Danqi Chen, Eunsol Choi, Minjoon Seo, Patrick Lewis, Robin Jia, and Sewon Min, editors, Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 42–50, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.mrqa-1.4. URL https://aclanthology.org/2021.mrqa-1.4/. ↗
  83. Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert. Ragas: Automated Evaluation of Retrieval Augmented Generation, April 2025. find
  84. Eirini Papagiannopoulou and Grigorios Tsoumakas. A review of keyphrase extraction. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(2):e1339, 2020. ISSN 1942-4787. find
  85. Mingyang Song, Yi Feng, and Liping Jing. A survey on recent advances in keyphrase extraction from pre-trained language models. Findings of the association for computational linguistics: EACL 2023, pages 2153–2164, 2023. find
  86. Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. Deep keyphrase generation. pages 582–592, 2017.
  87. Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. pages 404–411, 2004. find
  88. Ricardo Campos, Vítor Mangaravite, Arian Pasquali, Alípio Jorge, Célia Nunes, and Adam Jatowt. YAKE! Keyword extraction from single documents using multiple local features. Information Sciences, 509:257–289, 2020. ISSN 0020-0255. find
  89. Maarten Grootendorst, Heinz-Alexander Fuetterer, F Luca, Anurag Dhadse, Art Matsak, Igor Pechersky, Priyanshul Govil, Sam Frampton, and Yuki Ogura. MaartenGr/KeyBERT: v0. 9. Zenodo, 2021. find
  90. Roberto Martínez-Cruz, Alvaro J. López-López, and José Portela. ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task, June 2023. ↗
  91. Mingyang Song, Haiyun Jiang, Shuming Shi, Songfang Yao, Shilong Lu, Yi Feng, Huafeng Liu, and Liping Jing. Is ChatGPT A Good Keyphrase Generator? A Preliminary Study, December 2023. find
  92. David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning research, 3(Jan):993–1022, 2003. ↗
  93. Maarten Grootendorst. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794, 2022. ↗
  94. Anette Hulth. Improved automatic keyword extraction given more linguistic knowledge. pages 216–223, 2003.
  95. Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. Automatic keyphrase extraction from scientific articles. Language resources and evaluation, 47(3):723–742, 2013. ISSN 1574-020X. ↗
  96. Ygor Gallina, Florian Boudin, and Beatrice Daille. Kptimes: A large-scale dataset for keyphrase generation on news documents. In Proceedings of the 12th International Conference on Natural Language Generation, pages 130–135, 2019. find
  97. Olivier Salaün, Frédéric Piedboeuf, Guillaume Le Berre, David Alfonso-Hermelo, and Philippe Langlais. Europa: a legal multilingual keyphrase generation dataset. pages 12718–12736, 2024. find
  98. Di Wu, Da Yin, and Kai-Wei Chang. Kpeval: towards fine-grained semantic-based keyphrase evaluation. pages 1959–1981, 2024. ↗
  99. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On Faithfulness and Factuality in Abstractive Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.173. ↗
  100. Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the Factual Consistency of Abstractive Text Summarization, October 2019.
  101. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076– 12100, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.741. ↗
  102. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Delong Chen, Wenliang Dai, Ho Shu Chan, Andrea Madotto, and Pascale Fung. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12):1–38, December 2023. ISSN 0360-0300, 1557-7341. doi: 10.1145/3571730. ↗
  103. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems, 43(2):1–55, March 2025. ISSN 1046-8188, 1558-2868. doi: 10.1145/3703155. ↗
  104. Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, and Pascale Fung. HalluLens: LLM Hallucination Benchmark, April 2025.
  105. Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and Policy Considerations for Deep Learning in NLP. In Anna Korhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1355. URL https://aclanthology.org/P19-1355/. ↗
  106. David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training, 2021. URL https://arxiv.org/abs/ 2104.10350. ↗
  107. Yanran Wu, Inez Hua, and Yi Ding. Unveiling environmental impacts of large language model serving: A functional unit view. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10560–10576, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.519. URL https://aclanthology.org/2025.acl-long. 519/. ↗
  108. Nidhal Jegham, Marwan Abdelatti, Chan Young Koh, Lassad Elmoubarki, and Abdeltawab Hendawi. How hungry is ai? benchmarking energy, water, and carbon footprint of llm inference, 2025. URL https://arxiv.org/abs/2505.09598. ↗
  109. Benoit Courty, Victor Schmidt, Sasha Luccioni, et al. CodeCarbon, 2024. URL https://zenodo.org/records/11171501. ↗
  110. Samuel Rincé and Adrien Banse. EcoLogits: Evaluating the Environmental Impacts of Generative AI. Journal of Open Source Software, 10(111):7471, 2025. doi: 10.21105/joss.07471. URL https://joss.theoj.org/papers/10.21105/ joss.07471. ↗
  111. Transparenz von KI-Systemen. Technical report, BSI, July 2024. find
  112. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pages 220–229, 2019. find
  113. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12):86–92, 2021. ↗
  114. Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, and Percy Liang. The foundation model transparency index. arXiv preprint arXiv:2310.12941, 2023. ↗
  115. Alexander Wan, Kevin Klyman, Sayash Kapoor, Nestor Maslej, Shayne Longpre, Betty Xiong, Percy Liang, and Rishi Bommasani. The 2025 foundation model transparency index. arXiv preprint arXiv:2512.10169, 2025. ↗
  116. European Commission. Contents code – gpai, 2026. URL https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai. ↗
  117. Thilo Hagendorff. On the inevitability of left-leaning political bias in aligned language models, 2025. URL https://arxiv.org/abs/2507.15328. preprint. ↗
  118. Luca Rettenberger, Markus Reischl, and Mark Schutera. Assessing political bias in large language models. Journal of Computational Social Science, 8(2): 42, 2025. ISSN 2432-2725. doi: 10.1007/s42001-025-00376-w. URL https://doi.org/10.1007/s42001-025-00376-w. ↗
  119. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15295–15311, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. acl-long.816. URL https://aclanthology.org/2024.acl-long.816/. ↗
  120. Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation, 2023. URL https://arxiv.org/abs/2301.01768. ↗
  121. Chadi Helwe, Oana Balalau, and Davide Ceolin. Navigating the political compass: Evaluating multilingual LLMs across languages and nationalities. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, pages 17179–17204, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl. 883. URL https://aclanthology.org/2025.findings-acl.883/. ↗
  122. David Rozado. The political biases of chatgpt. Social Sciences, 12(3), 2023. ISSN 2076-0760. doi: 10.3390/socsci12030148. URL https://www.mdpi.com/ 2076-0760/12/3/148. ↗
  123. Jan Batzner, Volker Stocker, Stefan Schmid, and Gjergji Kasneci. Germanpartiesqa: Benchmarking commercial large language models and ai companions for political alignment and sycophancy. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(1):330–342, Oct. 2025. doi: 10.1609/aies.v8i1. 36552. URL https://ojs.aaai.org/index.php/AIES/article/view/36552. ↗
  124. Riccardo Lunardi, David La Barbera, and Kevin Roitero. The elusiveness of detecting political bias in language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM ’24, page 3922–3926, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704369. doi: 10.1145/3627673.3680002. URL https://doi.org/10.1145/3627673.3680002. ↗
  125. Yuntao Bai et al. Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022. URL https://arxiv.org/abs/2204.05862. ↗
  126. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems, February 2020. URL http://arxiv.org/abs/1905.00537. arXiv:1905.00537 [cs]. ↗
  127. A Survey on Evaluation of Large Language Models. 15. ISSN 2157-6904, 2157-6912. doi: 10.1145/3641289. URL https://dl.acm.org/doi/10.1145/3641289. ↗
  128. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems, November 2021. URL http://arxiv.org/abs/2110.14168. arXiv:2110.14168 [cs]. ↗
  129. Isabelle Mohr, Markus Krimmel, Saba Sturua, Mohammad Kalim Akram, Andreas Koukounas, Michael Günther, Georgios Mastrapas, Vinit Ravishankar, Joan Fontanals Martínez, Feng Wang, et al. Multi-task contrastive learning for 8192-token bilingual text embeddings. arXiv preprint arXiv:2402.17016, 2024. ↗
  130. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759, 2016. ↗
  131. European Commission. General-purpose AI code of practice — transparency chapter. https://ec.europa.eu/newsroom/dae/redirection/document/118120, July 2025. Accessed: 2026-05-11. ↗
  132. Raphael Scheible, Johann Frei, Fabian Thomczyk, Henry He, Patric Tippmann, Jochen Knaus, Victor Jaravine, Frank Kramer, and Martin Boeker. GottBERT: a pure German language model. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21237–21250, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024.emnlp-main.1183. URL https://aclanthology.org/2024.emnlp-main.1183/. ↗
  133. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL https://arxiv.org/ abs/1907.11692. ↗
  134. Project Apertus, Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pasztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Ďurech, et al. Apertus: Democratizing open and compliant llms for global language environments. arXiv preprint arXiv:2509.14233, 2025. ↗
  135. K. Krippendorff. Content Analysis: An Introduction to Its Methodology. SAGE Publications, Thousand Oaks, CA, USA, 2013. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dalerci-2026-m-ove-holistic-llm-benchmark/