Elamur Research⌕
☰
home / papers / 2025 / SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks

SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks

Paper · 2025 · Yilun Zhao, Kaiyan Zhang, Tiansheng Hu et al.

Summary

Judging how well foundation models answer open-ended research questions that rely on scientific papers is hard. Expert-built benchmarks are costly, fixed in time and quickly outdated. Automatic LLM judges often miss the subtle domain requirements of science. SciArena tackles this with a public, Chatbot Arena–style site. Researchers pose real questions, and a retrieval pipeline adapted from Ai2's Scholar QA gathers relevant papers. Two randomly sampled models then write long-form answers with citations, and the user picks the better one. The resulting votes feed an Elo leaderboard. During its first eight months, the platform gathered more than 20,000 researcher votes on 47 models. The authors filtered these votes for quality and checked annotator agreement. o3, Claude-4.1 models and GPT-5 lead the ranking. From the vote data the authors also derived SciArena-Eval, a test of whether LLM judges can reproduce human preferences. Even the strongest judge reaches only modest accuracy, which signals that automated evaluation of literature-grounded answers is still unreliable. All data, code and the benchmark are open (NeurIPS 2025 Datasets and Benchmarks).

Key points

  • Retrieval pipeline: It rephrases and decomposes queries, then searches abstracts (>100M papers) and snippets (~11.7M papers). A cross-encoder reranks the results, and the top 30 passages go to the two sampled models.
  • Leaderboard: The top Elo scores are o3 (1151.4), Claude-4.1-Opus (1126.4) and GPT-5 (1122.5).
  • SciArena-Eval: The benchmark holds 2,000 pairwise examples, 500 per discipline and balanced between A and B wins, with ties removed. The best judge, o3, scores 65.1% against a 50% random baseline. Gemini-2.5-Flash-Preview and the Llama-4 models barely beat chance.
  • Reasoning helps judging: Reasoning variants beat their non-reasoning siblings, for example o4-mini over GPT-4.1 (+2.9 points) and DeepSeek-R1 over DeepSeek-V3 (+0.7 points).
  • Bias controls: The platform includes only publicly available models and filters out anomalous voters. Voters favored relevant, correctly attributed citations, and the number of citations had little effect on their choices.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States353 works · 200,043 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓Harbin Institute of TechnologyHarbin · China158 works · 28,841 citations🎓Yale UniversityNew Haven · United States95 works · 12,765 citations🎓Nanjing UniversityNanjing · China74 works · 7,814 citations🎓New York University ShanghaiShanghai · China8 works · 523 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

Cite

Cite as Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Charles McGrady, Taira Anderson, Jonathan Bragg, Joseph Chee Chang, Jesse Dodge, Matt Latzke, Yixin Liu, Xiangru Tang, Zihang Wang, Chen Zhao, Hannaneh Hajishirzi, Doug Downey and Arman Cohan. 2025. SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks. arXiv:2507.01001.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2507.01001 · arXiv 2507.01001

@misc{zhao2025sciarena,
  title = {{SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks}},
  author = {Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan {Le Bras}, Charles McGrady, Taira Anderson, Jonathan Bragg, Joseph Chee Chang, Jesse Dodge, Matt Latzke, Yixin Liu, Xiangru Tang, Zihang Wang, Chen Zhao, Hannaneh Hajishirzi, Doug Downey and Arman Cohan},
  year = {2025},
  doi = {10.48550/arxiv.2507.01001},
  eprint = {2507.01001},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2507.01001}
}

References

90 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ai2. Olmo 3: Charting a path through the model flow to lead open-source ai. https://allenai.org/blog/olmo3, November 2025. ↗
  2. Anirudh Ajith, Mengzhou Xia, Alexis Chevalier, Tanya Goyal, Danqi Chen, and Tianyu Gao. LitSearch: A retrieval benchmark for scientific literature search. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces…
  3. Bridger Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu A. Ha, Rodney Michael Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler C. Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna L. Power, Sam Skjonsberg, Lu… — Construction of the Literature Graph in Semantic Scholar
  4. Anthropic. Claude 3.7 sonnet and claude code. claude-3-7-sonnet, February 2025. https://www.anthropic.com/news/ ↗
  5. Anthropic. Claude opus 4.1. https://www.anthropic.com/news/claude-opus-4-1/, August 2025. ↗
  6. Anthropic. Introducing claude 4. https://www.anthropic.com/news/claude-4, May 2025. ↗
  7. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D’arcy, David Wadden, Matt Latzke, Minyang Tian, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Graham Neubig, Dan Weld, Doug Downey, Wen tau… find
  8. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D’Arcy, David Wadden, Matt Latzke, Minyang Tian, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke S. Zettlemoyer, Graham Neubig, Dan Weld, Doug Downey, Wen t… find
  9. Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, Pengfei Liu, and Graham Neubig. Re-evaluating evaluation in text summarization. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 93…
  10. Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952. find
  11. Isabel Cachola, Kyle Lo, Arman Cohan, and Daniel Weld. TLDR: Extreme summarization of scientific documents. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4766–4777, Online, November 2020. Association for Computational Li…
  12. Hui Chen, Miao Xiong, Yujie Lu, Wei Han, Ailin Deng, Yufei He, Jiaying Wu, Yibo Li, Yue Liu, and Bryan Hooi. Mlr-bench: Evaluating ai agents on open-ended machine learning research, 2025. find
  13. Cheng-Han Chiang and Hung yi Lee. Can large language models be an alternative to human evaluations? In Annual Meeting of the Association for Computational Linguistics, 2023. find
  14. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: An open platform for evaluating LLMs by human preference. In Forty-first International Conference on Machine L…
  15. Christopher Chou, Lisa Dunlap, Koki Mashita, Krishna Mandal, Trevor Darrell, Ion Stoica, Joseph Gonzalez, and Wei-Lin Chiang. Visionarena: 230k real world user-vlm conversations with preference labels. ArXiv, abs/2412.08687, 2024. find
  16. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. SPECTER: Document-level representation learning using citation-informed transformers. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Compu…
  17. Consensus. Consensus – ai for research, 2024. Accessed: 2025-05-02. find
  18. Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakra…
  19. Google DeepMind. Gemini – deep research mode, 2024. Accessed: 2025-05-02. find
  20. DeepSeek-AI. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, December 2024. find
  21. DeepSeek-AI. Deepseek-r1-0528 release. news250528/, May 2025. https://api-docs.deepseek.com/news/ ↗
  22. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
  23. Yann Dubois, Percy Liang, and Tatsunori Hashimoto. Length-controlled alpacaeval: A simple debiasing of automatic evaluators. In First Conference on Language Modeling, 2024. find
  24. Lisa Dunlap, Krishna Mandal, Trevor Darrell, Jacob Steinhardt, and Joseph E Gonzalez. Vibecheck: Discover and quantify qualitative differences in large language models, 2025. find
  25. Elicit. Elicit – the ai research assistant, 2024. Accessed: 2025-05-02. find
  26. Arpad E Elo. The proposed uscf rating system, its development, theory, and applications. Chess life, 22(8):242–247, 1967. find
  27. Ronald Aylmer Fisher. Statistical methods for research workers. Number 5. Oliver and Boyd, 1928. find
  28. Google. Gemini 2.5: Our most intelligent ai model. https://blog.google/technology/ google-deepmind/gemini-model-thinking-updates-march-2025//, March 2025. ↗
  29. Google. Gemini 2.5 pro preview: even better coding performance. https://developers.googleblog.com/en/gemini-2-5-pro-io-improved-coding-performance/, April 2025. ↗
  30. Google. Gemini 3 pro: Best for complex tasks and bringing creative concepts to life. https://deepmind.google/models/gemini/pro/, November 2025. ↗
  31. Google. Start building with gemini 2.5 flash. https://developers.googleblog.com/en/ start-building-with-gemini-25-flash/, April 2025. ↗
  32. Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, and Wenhu Chen. GenAI arena: An open evaluation platform for generative models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. find
  33. Tetsu Kasanishi, Masaru Isonuma, Junichiro Mori, and Ichiro Sakata. SciReviewGen: A large-scale dataset for automatic literature review generation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, pages 6695–6715, T… find
  34. Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu… find
  35. Yoonjoo Lee, Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Ho Hin Lee, and Moontae Lee. Qasa: Advanced question answering on scientific articles. In International Conference on Machine Learning, 2023. find
  36. Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, and Arman Cohan. M3SciQA: A multimodal multi-document scientific QA benchmark for evaluating foundation models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNL… find
  37. Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline, 2024.
  38. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval, 5 2023. ↗
  39. Gabrielle Kaili-May Liu, Bowen Shi, Avi Caciularu, Idan Szpektor, and Arman Cohan. Mdcure: A scalable pipeline for multi-document instruction-following, 2025.
  40. Haokun Liu, Sicong Huang, Jingyu Hu, Yangqiaoyu Zhou, and Chenhao Tan. Hypobench: Towards systematic and principled benchmarking for hypothesis generation, 2025. find
  41. Hongyi Liu, Qingyun Wang, Payam Karisani, and Heng Ji. Named entity recognition under domain shift via metric learning for life sciences. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lin… find
  42. Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki,…
  43. Yixin Liu, Alexander Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq Joty, Pengfei Liu, Dragomir Radev, Chien-Sheng Wu, and Arman Cohan. Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization. In Kevin Duh, Helena Gomez, and Steven… find
  44. Yixin Liu, Kejian Shi, Alexander Fabbri, Yilun Zhao, PeiFeng Wang, Chien-Sheng Wu, Shafiq Joty, and Arman Cohan. ReIFE: Re-evaluating instruction-following evaluation. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, NAACL. Association for Computational Linguistics, April 2025. find
  45. Yao Lu, Yue Dong, and Laurent Charlin. Multi-XScience: A large-scale dataset for extreme multi-document summarization of scientific articles. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)…
  46. Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, and Bill Yuchen Lin. Wildvision: Evaluating vision-language models in the wild with human preferences. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. find
  47. Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. ExpertQA: Expert-curated questions and attributed answers. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computa…
  48. Meta AI. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai.meta.com/blog/llama-4-multimodal-intelligence/, April 2025. ↗
  49. MINIMAX. Minimax-m1, the world’s first open-source, large-scale, hybrid-attention reasoning model. https://www.minimax.io/news/minimaxm1/, June 2025. ↗
  50. Ministral AI. Medium is the new large. https://mistral.ai/news/mistral-medium-3/, May 2025. ↗
  51. Mihran Miroyan, Tsung-Han Wu, Logan King, Tianle Li, Jiayi Pan, Xinyan Hu, Wei-Lin Chiang, Anastasios N. Angelopoulos, Trevor Darrell, Narges Norouzi, and Joseph E. Gonzalez. Search arena: Analyzing search-augmented llms, 2025. find
  52. Mistral AI Team. Mistral small 3. https://mistral.ai/news/mistral-small-3/, January 2025. ↗
  53. MoonshotAI. Kimi k2: Open agentic intelligence. https://moonshotai.github.io/Kimi-K2//, July 2025. ↗
  54. mrfakename, Vaibhav Srivastav, Clémentine Fourrier, Lucain Pouget, Yoach Lacombe, main, and Sanchit Gandhi. Text to speech arena. https://huggingface.co/spaces/TTS-AGI/TTS-Arena, 2024. ↗
  55. OpenAI. Chatgpt – deep research mode, 2024. Accessed: 2025-05-02. find
  56. OpenAI. Gpt-5.1: A smarter, more conversational chatgpt. gpt-5-1/, November 2025. https://openai.com/index/ ↗
  57. OpenAI. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/, April 2025. ↗
  58. OpenAI. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/, August 2025. ↗
  59. OpenAI. Introducing gpt-oss. https://openai.com/index/introducing-gpt-oss//, August 2025. ↗
  60. OpenAI. Introducing openai o3 and o4-mini. introducing-o3-and-o4-mini/, April 2025. https://openai.com/index/ ↗
  61. Perplexity. Perplexity ai – ask anything, 2024. Accessed: 2025-05-02. find
  62. Qwen Team. Qwen3: Think deeper, act faster, April 2025. find
  63. Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. find
  64. Scale AI. Advancing frontier model evaluation. Scale AI Blog, 2024. find
  65. Scite. Scite – smart citations for research, 2024. Accessed: 2025-05-02. find
  66. Aamir Shakir, Darius Koenig, Julius Lipp, and Sean Lee. Boost your search with the crispy mixedbread rerank models, 2024. find
  67. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers. In The Thirteenth International Conference on Learning Representations, 2025.
  68. Amanpreet Singh, Joseph Chee Chang, Chloe Anastasiades, Dany Haddad, Aakanksha Naik, Amber Tanaka, Angele Zamarron, Cecile Nguyen, Jena D. Hwang, Jason Dunkleberger, Matt Latzke, Smita Rao, Jaron Lochner, Rob Evans, Rodney Kinney, Daniel S. Weld, Doug Downey, and Sergey Feldman. Ai2 scholar qa: Orga… find
  69. Amanpreet Singh, Joseph Chee Chang, Chloe Anastasiades, Dany Haddad, Aakanksha Naik, Amber Tanaka, Angele Zamarron, Cecile Nguyen, Jena D. Hwang, Jason Dunkleberger, Matt Latzke, Smita R Rao, Jaron Lochner, Rob Evans, Rodney Kinney, Daniel S. Weld, Doug Downey, and Sergey Feldman. Ai2 scholar qa: Or… find
  70. Amanpreet Singh, Mike D’Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. SciRepEval: A multi-format benchmark for scientific document representations. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pag… find
  71. Shivalika Singh, Yiyang Nan, Alex Wang, Daniel D’souza, Sayash Kapoor, A. Ustun, Sanmi Koyejo, Yuntian Deng, Shayne Longpre, Noah Smith, Beyza Hilal Ermiş, Marzieh Fadaee, and Sara Hooker. The leaderboard illusion. 2025. find
  72. Shruti Singh, Nandan Sarkar, and Arman Cohan. SciDQA: A deep reading comprehension dataset over scientific papers. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20908–20923, Miami, Florida,… find
  73. Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A. Metoyer. Limitations of the llm-as-a-judge approach for evaluating llm outputs in expert knowledge tasks, 2024. find
  74. Aryan Vichare, Anastasios N. Angelopoulos, Wei-Lin Chiang, Kelly Tang, and Luca Manolache. Webdev arena: A live llm leaderboard for web app development, 2025. find
  75. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. Fact or fiction: Verifying scientific claims. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pr…
  76. Bowen Wang, Xinyuan Wang, Jiaqi Deng, Tianbao Xie, Ryan Li, Yanzhe Zhang, Gavin Li, Toh Jing Hua, Ion Stoica, Wei-Lin Chiang, Diyi Yang, Yu Su, Yi Zhang, Zhiguo Wang, Victor Zhong, and Tao Yu. Computer agent arena: Compare and test computer use agents on crowdsourced real-world tasks, 2025. find
  77. Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan, and Yilun Zhao. Sciver: Evaluating foundation models for multimodal scientific claim verification, 2025.
  78. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. SciMON: Scientific inspiration machines optimized for novelty. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 279–299,… find
  79. xAI. Grok 3 beta — the age of reasoning agents. https://x.ai/news/grok-3, February 2025. ↗
  80. xAI. Grok 4. https://x.ai/news/grok-4, July 2025. ↗
  81. Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig, and Arman Cohan. Can LLMs identify critical limitations within scientific research? a systematic evaluation on AI research papers. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd…
  82. Nithik Yekollu, Arth Bohra, Ashwin Chirumamilla, Kai Wen, Sai Kolasani Wei-Lin Chiang, Anastasios Angelopoulos, Joseph E. Gonzalez, Ion Stoica, and Shishir G. Patil. Agent arena. 2024. find
  83. You.com. You.com – personalized ai search, 2024. Accessed: 2025-05-02. find
  84. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. In The Twelfth International Conference on Learning Representations, 2024. find
  85. Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang, and Jiawei Han. A comprehensive survey of scientific large language models and their applications in scientific discovery. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirica… find
  86. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations, 2024.
  87. Yilun Zhao, Weiyuan Chen, Zhijian Xu, Manasi Patwardhan, Chengye Wang, Yixin Liu, Lovekesh Vig, and Arman Cohan. AbGen: Evaluating large language models in ablation study design and evaluation for scientific research. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, ed…
  88. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Haotong Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. ArXiv, abs/2306.05685, 2023.
  89. Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, and Pengfei Liu. OpenResearcher: Unleashing AI for accelerated scientific research. In Delia Irazu Hernandez Farias, Tom… find
  90. Zhipu ai. Glm-4.5: Reasoning, coding, and agentic abililties. https://z.ai/blog/glm-4.5//, July 2025. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2025-sciarena-open-evaluation-platform-non/