Elamur Research⌕
☰
home / papers / 2025 / Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation

Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation

Paper · 2025 · Steffen Eger, Yong Cao, Jennifer D'Souza et al.

Summary

The survey looks at how large (multimodal) language models are starting to reshape scientific work. Earlier reviews tended to cover single disciplines or a few tasks. This paper instead follows the research lifecycle from start to finish. It covers five stages: finding and summarizing literature, generating ideas and running experiments, producing text such as titles, abstracts and citations, creating and interpreting figures, tables, slides and posters, and peer review. The authors chose a narrative rather than systematic method, so they pick representative work and compare methods across tasks instead of cataloguing everything. For each stage they discuss datasets, methods, evaluation practice, limitations and ethical risks, and a separate section addresses research integrity.

Their overall assessment is cautious. Current systems help with parts of the workflow, but their abilities are patchy. Many rely on narrow benchmarks and need close human supervision. For now, AI4Science should be treated as a set of supporting tools, not a replacement for human expertise. The survey is useful as an orientation map for newcomers and for teams building AI-for-science pipelines.

Key points

  • Evidence of adoption: About 148,000 papers from 22 non-CS fields show sharply rising citations of LLMs between 2018 and 2024.
  • Example tools covered: Elicit and ORKG ASK (search), The AI Scientist (experimentation), and AutomaTikZ and DeTikZify (figure generation).
  • Assessment by stage:
    • Generated ideas tend to be novel but often infeasible.
    • Automated experimentation is still early-stage and error-prone.
    • Automated reviewing works as an assistant but not as a standalone judge.
  • Truthfulness benchmarks discussed: TruthfulQA, HaluEval, FELM, SelfAware, FreshQA, Pinocchio and TrustLLM. TrustLLM evaluations favour proprietary models, with Llama2 as a notable exception, and show gains when models are given external knowledge.
  • Community resources: New workshops include NSLP, FM4Science, AISD, AI4Research, Agents4Science and SciProdLLM. The authors maintain an updated resource list at github.com/NL2G/TransformingScienceLLMs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of SheffieldSheffield · United Kingdom113 works · 21,542 citations🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🎓University of TübingenTübingen · Germany75 works · 41,081 citations🎓Universität HamburgHamburg · Germany79 works · 5,422 citations🎓University of AberdeenAberdeen · United Kingdom20 works · 1,385 citations🎓University of ManitobaWinnipeg · Canada5 works · 2,378 citations🔬Austrian Research Institute for Artificial IntelligenceVienna · Austria4 works · 27 citations🎓Interdisciplinary Transformation University AustriaLinz · Austria5 works · 35 citations🎓University of Technology NurembergNuremberg · Germany6 works · 30 citations

Also on the byline: Technische Informationsbibliothek (TIB).

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Steffen Eger, Yong Cao, Jennifer D'Souza, Andreas Geiger, Christian Greisinger, Stephanie Gross, Yufang Hou, Brigitte Krenn, Anne Lauscher, Yizhi Li, Chenghua Lin, Nafise Sadat Moosavi, Wei Zhao and Tristan Miller. 2025. Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation. arXiv:2502.05151.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2502.05151 · arXiv 2502.05151

@misc{eger2025transforming,
  title = {{Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation}},
  author = {Steffen Eger, Yong Cao, Jennifer D'Souza, Andreas Geiger, Christian Greisinger, Stephanie Gross, Yufang Hou, Brigitte Krenn, Anne Lauscher, Yizhi Li, Chenghua Lin, Nafise Sadat Moosavi, Wei Zhao and Tristan Miller},
  year = {2025},
  doi = {10.48550/arxiv.2502.05151},
  eprint = {2502.05151},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.05151}
}

References

317 references, 52 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbi Abdel-Rehim, Hector Zenil, Oghenejokpeme Orhobor, Marie Fisher, Ross J Collins, Elizabeth Bourne, Gareth W Fearnley, Emma Tate, Holly X Smith, Larisa N Soldatova, et al. 2025. Scientific hypothesis generation by large language models: laboratory validation in breast cancer treatment. J. Royal S… find
  2. Ahmed AbuRa’ed, Horacio Saggion, Alexander Shvets, and Àlex Bravo. 2020. Automatic related work section generation: experiments in scientific document abstracting. Scientometrics 125 (2020), 3159–3185. find
  3. Hassnian Ali and Ahmet Faruk Aysan. 2024. Ethical dimensions of generative AI: A cross-domain analysis using machine learning structural topic modeling. Int. J. Ethics Syst. 41, 1 (2024), 3–34. find
  4. Signe Altmäe, Alberto Sola-Leyva, and Andres Salumets. 2023. Artificial intelligence in scientific writing: a friend or a foe? Reproductive BioMedicine Online 47, 1 (2023), 3–9. find
  5. Maha Amami, Gabriella Pasi, Fabio Stella, and Rim Faiz. 2016. An lda-based approach to scientific paper recommendation. In Nat. Lang. Process. Inf. Syst. 21st Int. Conf. on Appl. Nat. Lang. to Inf. Syst. NLDB 2016. Springer, 200–210. find
  6. Isaac Ampomah, James Burton, Amir Enshaei, and Noura Al Moubayed. 2022. Generating Textual Explanations for Machine Learning Models Performance: A Table-to-Text Task. In Proc. 13th Lang. Resour. Eval. Conf., Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Th… find
  7. Nash Anderson, Daniel L. Belavy, Stephen M. Perle, Sharief Hendricks, Luiz Hespanhol, Evert Verhagen, and Aamir R. Memon. 2023. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in Sports & Exercise Medicine ma… find
  8. Ewa Andrejczuk, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, and Yasemin Altun. 2022. Table-to-text generation and pre-training with TabT5. In Find. Assoc. for Comput. Linguist. EMNLP 2022, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). 6758–6766. find
  9. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Tian, D’arcy Mike, David Wadden, Matt Latzke, Minyang, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Dan Weld, Graham Neubig, Doug Downey, Wen-tau… find
  10. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. In The Twelfth Int. Conf. on Learn. Represent..
  11. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2025. ResearchAgent: Iterative research idea generation over scientific literature with large language models. In Proc. 2025 Conf. Nations Am. Chapter Assoc. for Comput. Linguist.. 6709–6738.
  12. Jiaxin Bai, Yicheng Wang, Tianshi Zheng, Yue Guo, Xin Liu, and Yangqiu Song. 2024. Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation. In Proc. 62nd Annu. Meet. Assoc. for Comput. Linguist., Vol. 1. 1312–1329. find
  13. Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs. In The Thirteen. Int. Conf. on Learn. Represent..
  14. Maria Teresa Baldassarre, Danilo Caivano, Berenice Fernandez Nieto, Domenico Gigante, and Azzurra Ragone. 2023. The social impact of generative AI: An analysis on ChatGPT. In Proc. 2023 ACM Conf. on Inf. Technol. for Soc. Good. 363–373. find
  15. Sambaran Bandyopadhyay, Himanshu Maheshwari, Anandhavelu Natarajan, and Apoorv Saxena. 2024. Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach. In Proc. 17th Int. Nat. Lang. Gener. Conf.. 222–229. find
  16. Sachin Banker, Promothesh Chatterjee, Himanshu Mishra, and Arul Mishra. 2024. Machine-assisted social psychology hypothesis generation. Am. Psychol. 79, 6 (2024), 789. Manuscript submitted to ACM find
  17. Trapit Bansal, David Belanger, and Andrew McCallum. 2016. Ask the GRU: Multi-task learning for deep text recommendations. In Proc. 10th ACM Conf. on Recomm. Syst.. 107–114. find
  18. Alberto Bartoli and Eric Medvet. 2020. Exploring the Potential of GPT-2 for Generating Fake Reviews of Research Papers. In Proc. FSDM 2020, Antonio J. Tallón-Ballesteros (Ed.). 390–396. find
  19. Jonas Belouadi, Eddy Ilg, Margret Keuper, Hideki Tanaka, Masao Utiyama, Raj Dabre, Steffen Eger, and Simone Ponzetto. 2025. TikZero: Zero-shot text-guided graphics program synthesis. In Proc. IEEE/CVF Int. Conf. on Comput. Vis.. 17793–17806. find
  20. Jonas Belouadi, Anne Lauscher, and Steffen Eger. 2024. AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ. In Proc. Twelfth Int. Conf. on Learn. Represent..
  21. Jonas Belouadi, Simone Paolo Ponzetto, and Steffen Eger. 2024. DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ. In The Thirty-eighth Annu. Conf. on Neural Inf. Process. Syst.. find
  22. Lamia Ben Ezzdine, Wissem Dhahbi, Ismail Dergaa, Halil İbrahim Ceylan, Noomen Guelmami, Helmi Ben Saad, Karim Chamari, Valentina Stefanica, and Abdelfatteh El Omri. 2025. Physical activity and neuroplasticity in neurodegenerative disorders: a comprehensive review of exercise interventions, cognitive… find
  23. Chandra Bhagavatula, Sergey Feldman, Russell Power, and Waleed Ammar. 2018. Content-Based Citation Recommendation. In Proc. 2018 Conf. North Am. Chapter Assoc. for Comput. Linguist., Marilyn Walker, Heng Ji, and Amanda Stent (Eds.). 238–251. find
  24. Prabhat Bharti, Meith Navlakha, Mayank Agarwal, and Asif Ekbal. 2023. PolitePEER: Does peer review hurt? A dataset to gauge politeness intensity in the peer reviews. Lang. Resour. Eval. (2023), 1–23. find
  25. Prabhat Kumar Bharti, Shashi Ranjan, Tirthankar Ghosal, Mayank Agrawal, and Asif Ekbal. 2021. PEERAssist: Leveraging on Paper-Review Interactions to Predict Peer Review Decisions. In Towards Open Trust. Digit. Soc.. 421–435. find
  26. Christine L. Borgman. 2007. Scholarship in the Digital Age: Information, Infrastructure, and the Internet. MIT Press. find
  27. Lutz Bornmann, Robin Haunschild, and Rüdiger Mutz. 2021. Growth Rates of Modern Science: a Latent Piecewise Growth Curve Approach to Model Publication Numbers from Established and New Literature Databases. Humanit. Soc. Sci. Commun. 8, 224 (2021). find
  28. Jennifer A. Byrne. 2016. Improving the Peer Review of Narrative Literature Reviews. Res. Integr. Peer Rev. 1, 12 (2016). find
  29. Fengyu Cai, Xinran Zhao, Tong Chen, Sihao Chen, Hongming Zhang, Iryna Gurevych, and Heinz Koeppl. 2024. MixGR: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 10369–10391. find
  30. Andres Castellanos-Gomez. 2023. Good practices for scientific article writing with ChatGPT and other artificial intelligence language models. Nanomanufacturing 3, 2 (2023), 135–138. find
  31. Miaosen Chai, Emily Herron, Erick Cervantes, and Tirthankar Ghosal. 2024. Exploring Scientific Hypothesis Generation with Mamba. In Proc. 1st Workshop on NLP for Sci.. 197–207. find
  32. Souvic Chakraborty, Pawan Goyal, and Animesh Mukherjee. 2020. Aspect-based Sentiment Analysis of Scientific Reviews. In Proc. ACM/IEEE Jt. Conf. on Digit. Libr. 2020. 207–216. find
  33. Iain Chalmers, Larry V. Hedges, and Harris Cooper. 2002. A Brief History of Research Synthesis. Eval. & Health Prof. 25, 1 (2002), 12–37. find
  34. Chu Sern Joel Chan, Aakanksha Naik, Matthew Akamatsu, Hanna Bekele, Erin Bransom, Ian Campbell, and Jenna Sparks. 2024. Overview of the Context24 Shared Task on Contextualizing Scientific Claims. In Proc. Fourth Workshop on Sch. Document Process.. 12–21. find
  35. A. Checco, L. Bracciale, P. Loreti, S. Pinfield, and G. Bianchi. 2021. AI-assisted peer review. Humanit. Soc. Sci. Commun. 8, 1 (January 2021), 1–11. find
  36. Chen Chen, Ayman Maqsood, Zhuang Zhang, Xiaobing Wang, Linrui Duan, Huanhuan Wang, Tianyang Chen, Siyu Liu, Qiutong Li, Jingshan Luo, et al. 2024. The use of ChatGPT to generate experimentally testable hypotheses for improving the surface passivation of perovskite solar cells. Cell Reports Phys. Sci… find
  37. Hong Chen, Hiroya Takamura, and Hideki Nakayama. 2021. SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation. In Find. Assoc. for Comput. Linguist. EMNLP 2021, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). 1483–1492. find
  38. Kexin Chen, Jiamin Lu, Junyou Li, Xiaoran Yang, Yuyang Du, Kunyi Wang, Qiannuan Shi, Jiahui Yu, Lanqing Li, Jiezhong Qiu, et al. 2023. Chemist-X: Large language model-empowered agent for reaction condition recommendation in chemical synthesis. arXiv:2311.10776 — [arXiv:2311.10776](https://arxiv.org/abs/2311.10776) find
  39. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. FELM: Benchmarking Factuality Evaluation of Large Language Models. In Adv. Neural Inf. Process. Syst. 36, Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.)… find
  40. Yanran Chen and Steffen Eger. 2023. Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End. In Proc. 4th Workshop on Eval. Comp. NLP Syst., Daniel Deutsch, Rotem Dror, Steffen Eger, Yang Gao, Christoph Leiter, Juri Opitz, and Andreas Rücklé (Eds.). 62–84. find
  41. Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. 2024. ScienceAgentBench: Toward Rigorous Assessme… find
  42. Huzi Cheng, Bin Sheng, Aaron Lee, Varun Chaudhary, Atanas G Atanasov, Nan Liu, Yue Qiu, Tien Yin Wong, Yih-Chung Tham, and Ying-Feng Zheng. 2024. Have AI-Generated Texts from LLM Infiltrated the Realm of Scientific Writing? A Large-Scale Analysis of Preprint Platforms. bioRxiv (2024), 2024–03. find
  43. Liying Cheng, Lidong Bing, Qian Yu, Wei Lu, and Luo Si. 2020. APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning. In Proc. 2020 Conf. on Empir. Methods Nat. Lang. Process.. 7000–7011. Manuscript submitted to ACM find
  44. Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation. In Proc. 60th Annu. Meet. Assoc. for Comput. Linguist., Smaranda Muresan, Preslav Nakov,… find
  45. Yizhou Chi, Yizhang Lin, Sirui Hong, Duyi Pan, Yaying Fei, Guanghao Mei, Bangbang Liu, Tianqi Pang, Jacky Kwok, Ceyao Zhang, et al. 2024. SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning. arXiv:2410.17238 — [arXiv:2410.17238](https://arxiv.org/abs/2410.17238) find
  46. Gautam Choudhary, Natwar Modani, and Nitish Maurya. 2021. ReAct: A Review Comment Dataset for Actionability (and more). In Web Inf. Syst. Eng. – WISE 2021. 336–343. find
  47. John Clement. 1989. Learning via Model Construction and Criticism: Protocol Evidence on Sources of Creativity in Science. In Handbook of Creativity: Assessment, Theory and Research, John A. Glover, Royce R. Ronning, and Cecil R. Reynolds (Eds.). Plenum, 341–381. find
  48. John J. Clement. 2022. Multiple Levels of Heuristic Reasoning Processes in Scientific Model Construction. Front. Psychol. 13 (2022). find
  49. Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, and Daniel S Weld. 2019. Pretrained language models for sequential sentence classification. arXiv:1909.04054 find
  50. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level Representation Learning using Citation-informed Transformers. In Proc. 58th Annu. Meet. Assoc. for Comput. Linguist., Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). 2270–2282.
  51. Alessandro Conti, Enrico Fini, Paolo Rota, Yiming Wang, Massimiliano Mancini, and Elisa Ricci. 2024. Automatic benchmarking of large multimodal models via iterative experiment programming. arXiv:2406.12321 — [arXiv:2406.12321](https://arxiv.org/abs/2406.12321) find
  52. Cristina Cornelio, Sanjeeb Dash, Vernon Austel, Tyler R. Josephson, Joao Goncalves, Kenneth L. Clarkson, Nimrod Megiddo, Bachir El Khadir, and Lior Horesh. 2023. Combining Data and Theory for Derivable Scientific Discovery with AI-Descartes. Nat. Commun. 14, 1777 (2023). find
  53. Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for YouTube recommendations. In Proc. 10th ACM Conf. on Recomm. Syst.. 191–198. find
  54. Zhiqing Cui, Jiahao Yuan, Hanqing Wang, Yanshu Li, Chenxu Du, and Zhenglong Ding. 2025. Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation. In Proc. 33rd ACM Int. Conf. on Multimed.. 5050–5059. find
  55. Mike D’Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. 2024. MARG: Multi-Agent Review Generation for Scientific Papers. arXiv:2401.04259 find
  56. Mike D’Arcy, Alexis Ross, Erin Bransom, Bailey Kuehl, Jonathan Bragg, Tom Hope, and Doug Downey. 2023. ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews. arXiv:2306.12587
  57. Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song. 2024. Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 9300–9322. find
  58. Zekun Deng, Zixin Zeng, Weiye Gu, Jiawen Ji, and Bolin Hua. 2021. Automatic Related Work Section Generation by Sentence Extraction and Reordering. In Proc. 1st Workshop on AI + Informetrics. find
  59. Ismail Dergaa, Feten Fekih-Romdhane, Jordan M Glenn, Mohamed Saifeddin Fessi, Karim Chamari, Wissem Dhahbi, Makram Zghibi, Nicola Luigi Bragazzi, Mohamed Ben Aissa, Noomen Guelmami, et al. 2023. Moving beyond the stigma: understanding and overcoming the resistance to the acceptance and adoption of a… find
  60. Danilo Dessí, Francesco Osborne, Diego Reforgiato Recupero, Davide Buscaldi, and Enrico Motta. 2022. CS-KG: A large-scale knowledge graph of research entities and claims in computer science. In Int. Semantic Web Conf.. 678–696. find
  61. Alphaeus Dmonte, Roland Oruche, Marcos Zampieri, Prasad Calyam, and Isabelle Augenstein. 2024. Claim Verification in the Age of Large Language Models: A Survey. arXiv:2408.14317
  62. Haoyu Dong, Mengkang Hu, Qinyu Xu, Haochen Wang, and Yue Hu. 2024. OpenTE: Open-Structure Table Extraction From Text. In IEEE Int. Conf. on Acoust. Speech Signal Process. ICASSP 2024. IEEE, 10306–10310. find
  63. Iddo Drori and Dov Te’eni. 2024. Human-in-the-Loop AI Reviewing: Feasibility, Opportunities, and Risks. J. Assoc. for Inf. Syst. 25, 1 (2024), 98–109. find
  64. John A. Drozdz and Michael R. Ladomery. 2024. The Peer Review Process: Past, Present, and Future. Br. J. Biomed. Sci. 81 (2024). find
  65. Jennifer D’Souza, Sören Auer, and Ted Pedersen. 2021. SemEval-2021 Task 11: NLPContributionGraph - Structuring Scholarly NLP Contributions for a Research Knowledge Graph. In Proc. 15th Int. Workshop on Semantic Eval. (SemEval-2021), Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aur… find
  66. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. NLPeer: A Unified Resource for the Computational Study of Peer Review. In Proc. 61st Annu. Meet. Assoc. for Comput. Linguist., Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.), Vol. 1. 5049–5073.
  67. Daniel Nygård Ege, Mark Goudswaard, James Gopsill, Ben Hicks, and Martin Steinert. 2024. IDEA Challenge 2022 Dataset: Prototypes from a Design Hackathon. Data Brief 54 (2024), 110363. doi:10.1016/j.dib.2024.110363 — [doi:10.1016/j.dib.2024.110363](https://doi.org/10.1016/j.dib.2024.110363) find
  68. Aryaz Eghbali and Michael Pradel. 2023. CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code. In Proc. 37th IEEE/ACM Int. Conf. on Autom. Softw. Eng.. Article 28. find
  69. Holly Else. 2023. Abstracts written by ChatGPT fool scientists. Nature 613 (2023), 423. find
  70. Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: A survey. J. Mach. Learn. Res. 20, 55 (2019), 1–21. find
  71. Faiza Farhat, Shahab Saquib Sohail, and Dag Øivind Madsen. 2023. How trustworthy is ChatGPT? The case of bibliometric analyses. Cogent Eng. 10, 1 (2023), 2222988. find
  72. Ronald A. Fisher. 1925. Statistical Methods for Research Workers. Oliver and Boyd. Manuscript submitted to ACM find
  73. Ronald A. Fisher. 1935. The Design of Experiments. Oliver and Boyd. find
  74. Michael Fromm, Evgeniy Faerman, Max Berrendorf, Siddharth Bhargava, Ruoxia Qi, Yao Zhang, Lukas Dennert, Sophia Selle, Yang Mao, and Thomas Seidl. 2021. Argument Mining Driven Analysis of Peer-Reviews. Proc. AAAI Conf. on Artif. Intell. 35, 6 (2021), 4758–4766. find
  75. Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. 2023. DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data. In Adv. Neural Inf. Process. Syst. 36, Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hard… find
  76. Tsu-Jui Fu, William Yang Wang, Daniel J. McDuff, and Yale Song. 2021. DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents. In AAAI Conf. on Artif. Intell.. find
  77. Martin Funkquist, Ilia Kuznetsov, Yufang Hou, and Iryna Gurevych. 2023. CiteBench: A Benchmark for Scientific Citation Text Generation. In Proc. 2023 Conf. on Empir. Methods Nat. Lang. Process.. 7337–7353. find
  78. Timur Galimzyanov, Sergey Titov, Yaroslav Golubev, and Egor Bogomolov. 2025. Drawing pandas: A benchmark for llms in generating plotting code. In 2025 IEEE/ACM 22nd Int. Conf. on Min. Softw. Repos.. IEEE, 503–507. find
  79. Catherine A. Gao, Frederick M. Howard, Nikolay S Markov, Emma C. Dyer, Siddhi Ramesh, Yuan Luo, and Alexander T. Pearson. 2023. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit. Med. 6, 1 (2023), 75. find
  80. Mingqi Gao, Xinyu Hu, Xunjian Yin, Jie Ruan, Xiao Pu, and Xiaojun Wan. 2025. LLM-based NLG evaluation: Current status and challenges. Comput. Linguist. (2025), 1–27.
  81. Andres Garcia-Silva, Cristian Berrio, and Jose Manuel Gomez-Perez. 2024. SPACE-IDEAS: A Dataset for Salient Information Detection in Space Innovation. In Proc. 2024 Jt. Int. Conf. on Comput. Linguist. Lang. Resour. Eval.. 15087–15092. find
  82. Eugene Garfield. 1955. Citation Indexes for Science. Science 122, 3159 (1955), 108–111. find
  83. Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, et al. 2025. Autopresent: Designing structured visuals from scratch. In Proc. Comput. Vis. Pattern Recognit. Conf.. 2902–2911. find
  84. Alireza Ghafarollahi and Markus J Buehler. 2025. SciAgents: automating scientific discovery through bioinspired multi-agent intelligent graph reasoning. Adv. Mater. 37, 22 (2025), 2413523. find
  85. Tirthankar Ghosal, Sandeep Kumar, Prabhat Kumar Bharti, and Asif Ekbal. 2022. Peer review analyze: A novel benchmark resource for computational analysis of peer reviews. PLOS One 17, 1 (01 2022), 1–29. find
  86. Tirthankar Ghosal, Kamal Kaushik Varanasi, and Valia Kordoni. 2022. HedgePeer: A dataset for uncertainty detection in peer reviews. In Proc. 22nd ACM/IEEE Jt. Conf. on Digit. Libr.. 1–5. find
  87. Tirthankar Ghosal, Rajeev Verma, Asif Ekbal, and Pushpak Bhattacharyya. 2019. DeepSentiPeer: Harnessing Sentiment in Review Texts to Recommend Peer Review Decisions. In Proc. 57th Annu. Meet. Assoc. for Comput. Linguist.. 1120–1130. find
  88. Hamed Babaei Giglou, Jennifer D’Souza, and Sören Auer. 2024. LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis. In 2024 ACM/IEEE Jt. Conf. on Digit. Libr.. 1–12. find
  89. Max Glockner, Yufang Hou, Preslav Nakov, and Iryna Gurevych. 2024. Grounding Fallacies Misrepresenting Scientific Publications in Evidence. arXiv:2408.12812 find
  90. Max Glockner, Yufang Hou, Preslav Nakov, and Iryna Gurevych. 2024. MISSCI: Reconstructing Fallacies in Misrepresented Science. In Proc. 62nd Annu. Meet. Assoc. for Comput. Linguist.. 4372–4405. find
  91. Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. 2025. Towards an AI co-scientist. arXiv:2502.18864
  92. Andrew V Gougherty and Hannah L Clipp. 2024. Testing the reliability of an AI-based large language model to extract ecological information from the scientific literature. npj Biodivers. 3, 1 (2024), 13. find
  93. Andrew Gray. 2024. ChatGPT “contamination”: estimating the prevalence of LLMs in the scholarly literature. arXiv:2403.16887 — [arXiv:2403.16887](https://arxiv.org/abs/2403.16887) find
  94. Christian Greisinger and Steffen Eger. 2026. TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning. In Proc. Fourteenth Int. Conf. on Learn. Represent.. In press. find
  95. Yanzhu Guo, Guokan Shang, Virgile Rennard, Michalis Vazirgiannis, and Chloé Clavel. 2023. Automatic Analysis of Substantiation in Scientific Peer Reviews. In Find. Assoc. for Comput. Linguist. EMNLP 2023. 10198–10216. find
  96. Thilo Hagendorff. 2024. Mapping the ethics of generative AI: A comprehensive scoping review. arXiv:2402.08323 — [arXiv:2402.08323](https://arxiv.org/abs/2402.08323) find
  97. James Hartley. 2008. Academic Writing and Publishing: A Practical Handbook. Routledge. find
  98. Zahra Hashemi, Zhiqiang Zhong, Jun Pang, and Wei Zhao. 2026. Malicious Repurposing of Open Science Artefacts by Using Large Language Models. arXiv:2601.18998 — [arXiv:2601.18998](https://arxiv.org/abs/2601.18998) find
  99. Janna Hastings. 2023. AI for Scientific Discovery. CRC PRess. find
  100. Jiyan He, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, Shuai Li, Jie Zhang, Kejiang Chen, Wenbo Zhou, Xing Xie, et al. 2023. Control risk for potential misuse of artificial intelligence in science. arXiv:2312.06632 — [arXiv:2312.06632](https://arxiv.org/abs/2312.06632) find
  101. Xin He, Kaiyong Zhao, and Xiaowen Chu. 2021. AutoML: A survey of the state-of-the-art. Knowledge-based Syst. 212 (2021), 106622. find
  102. Yichen He, Guanhua Huang, Peiyuan Feng, Yuan Lin, Yuchen Zhang, Hang Li, and Weinan E. 2025. PaSa: An LLM Agent for Comprehensive Academic Paper Search. In Proc. 63rd Annu. Meet. Assoc. for Comput. Linguist., Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). 11663–1…
  103. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. CLIPScore: A Reference-free Evaluation Metric for Image Captioning. In Proc. 2021 Conf. on Empir. Methods Nat. Lang. Process., Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). 7514–7528.
  104. Tony Hey, Stewart Tansley, and Kristin Tolle. 2009. Jim Gray on eScience: a Transformed Scientific Method. In The Fourth Paradigm: Data-Intensive Scientific Discovery, Tony Hey, Stewart Tansley, and Kristin Tolle (Eds.). Microsoft Research. find
  105. Cong Duy Vu Hoang and Min-Yen Kan. 2010. Towards Automated Related Work Summarization. In COLING 2010: Posters, Chu-Ren Huang and Dan Jurafsky (Eds.). 427–435. find
  106. Sally Hopewell, Mike Clarke, David Moher, Elizabeth Wager, Philippa Middleton, Douglas G. Altman, Kenneth F. Schulz, and Consort Group. 2008. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS medicine 5, 1 (2008), e20. find
  107. Yufang Hou, Charles Jochim, Martin Gleize, Francesca Bonin, and Debasis Ganguly. 2019. Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction. In Proc. 57th Annu. Meet. Assoc. for Comput. Linguist., Anna Korhonen, David Traum, and Lluís Màr… find
  108. Xiang Hu, Hongyu Fu, Jinge Wang, Yifeng Wang, Zhikun Li, Renjun Xu, Yu Lu, Yaochu Jin, Lili Pan, and Zhenzhong Lan. 2024. Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideas. arXiv:2410.14255 find
  109. Yue Hu and Xiaojun Wan. 2013. PPSGen: Learning to Generate Presentation Slides for Academic Papers. In Int. Jt. Conf. on Artif. Intell.. find
  110. Yue Hu and Xiaojun Wan. 2014. Automatic generation of related work sections in scientific papers: an optimization approach. In Proc. 2014 Conf. on Empir. Methods Nat. Lang. Process.. 1624–1633. find
  111. Xinyu Hua, Mitko Nikolov, Nikhil Badugu, and Lu Wang. 2019. Argument Mining for Understanding Peer Reviews. In Proc. 2019 Conf. North Am. Chapter Assoc. for Comput. Linguist.. 2131–2137. find
  112. Jie Huang and Kevin Chen-Chuan Chang. 2023. Citation: A key to building responsible and accountable large language models. arXiv:2307.02185 find
  113. Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2024. MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation. In Forty-first Int. Conf. on Mach. Learn..
  114. Shengzhi Huang, Qicong Wang, Wei Lu, Lingyu Liu, Zhenzhen Xu, and Yong Huang. 2025. PaperEval: A universal, quantitative, and explainable paper evaluation method powered by a multi-agent system. Inf. Process. & Manag. 62, 6 (2025), 104225. find
  115. Taesoon Hwang, Nishant Aggarwal, Pir Zarak Khan, Thomas Roberts, Amir Mahmood, Madlen M Griffiths, Nick Parsons, and Saboor Khan. 2024. Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts. Pl… find
  116. Joseph James, Chenghao Xiao, Yucheng Li, and Chenghua Lin. 2024. On the Rigour of Scientific Writing: Criteria, Analysis, and Insights. In Find. Assoc. for Comput. Linguist. EMNLP 2024. 6523–6538. find
  117. Peter Jansen, Oyvind Tafjord, Marissa Radensky, Pao Siangliulue, Tom Hope, Bhavana Dalvi, Bodhisattwa Prasad Majumder, Daniel S Weld, and Peter Clark. 2025. Codescientist: End-to-end semi-automated scientific discovery with code-based experimentation. In Find. Assoc. for Comput. Linguist. ACL 2025.… find
  118. Peiwen Jiang, Xinbo Lin, Zibo Zhao, Ruhui Ma, Yvonne Jie Chen, and Jinhua Cheng. 2024. TKGT: Redefinition and A New Way of Text-to-Table Tasks Based on Real World Demands and Knowledge Graphs Augmented LLMs. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 16112–16126. find
  119. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth Int. Conf. on Learn. Represent..
  120. Xia Jing, James J Cimino, Vimla L Patel, Yuchun Zhou, Jay H Shubrook, Chang Liu, and Sonsoles De Lacalle. 2024. Data-Driven Hypothesis Generation in Clinical Research: What We Learned from a Human Subject Study? Med. Res. Arch. 12, 2 (2024). find
  121. Léane Jourdan, Florian Boudin, Richard Dufour, Nicolas Hernandez, and Akiko Aizawa. 2025. ParaRev: Building a dataset for Scientific Paragraph Revision annotated with revision instruction. In Proc. First Workshop on Writ. Aids at Crossroads AI, Cogn. Sci. NLP. 35–44. find
  122. Léane Isabelle Jourdan, Florian Boudin, Nicolas Hernandez, and Richard Dufour. 2024. CASIMIR: A Corpus of Scientific Articles Enhanced with Multiple Author-Integrated Revisions. In Proc. 2024 Jt. Int. Conf. on Comput. Linguist. Lang. Resour. Eval., Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste,… find
  123. Samira Ebrahimi Kahou, Adam Atkinson, Vincent Michalski, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2018. FigureQA: An Annotated Figure Dataset for Visual Reasoning. arXiv:1710.07300 find
  124. Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications. In Proc. 2018 Conf. North Am. Chapter Assoc. for Comput. Linguist.. 1647–1661.
  125. SeongKu Kang, Yunyi Zhang, Pengcheng Jiang, Dongha Lee, Jiawei Han, and Hwanjo Yu. 2024. Taxonomy-guided Semantic Indexing for Academic Paper Search. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 7169–7184. find
  126. Wei-Yu Kao and An-Zi Yen. 2024. How We Refute Claims: Automatic Fact-Checking through Flaw Identification and Explanation. In Companion Proc. ACM on Web Conf. 2024. 758–761. find
  127. Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor, and Robert Stojnic. 2020. AxCell: Automatic Extraction of Results from Machine Learning Papers. In Proc. 2020 Conf. on Empir. Methods Nat. Lang. Process., Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu… find
  128. Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. A Diagram is Worth a Dozen Images. In Comput. Vision: ECCV 2016. 235–251.
  129. Neha Nayak Kennard, Tim O’Gorman, Rajarshi Das, Akshay Sharma, Chhandak Bagchi, Matthew Clinton, Pranay Kumar Yelugam, Hamed Zamani, and Andrew McCallum. 2022. DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions. In Proc. NAACL. 1234–1249. find
  130. M. M. Kessler. 1963. Bibliographic coupling between scientific papers. Am. Documentation 14, 1 (1963), 10–25. find
  131. Seong-Gon Kim. 2023. Using ChatGPT for language editing in scientific articles. Maxillofac. Plast. Reconstr. Surg. 45, 1 (2023), 13. find
  132. William R. King and Jun He. 2005. Understanding the Role and Methods of Meta-Analysis in IS Research. Commun. Assoc. for Inf. Syst. 16 (2005), 665–686. find
  133. Roger E. Kirk. 2009. Experimental Design. In The SAGE Handbook of Quantitative Methods in Psychology, Roger E. Millsap and Alberto Maydeu-Olivares (Eds.). 23–45. find
  134. Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause. 2024. Delving into ChatGPT usage in academic writing through excess vocabulary. arXiv:2406.07016 find
  135. Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. 2024. Tree Search for Language Model Agents. arXiv:2407.01476 find
  136. Buse Sibel Korkmaz and Antonio Del Rio Chanona. 2024. Integrating Table Representations into Large Language Models for Improved Scholarly Document Comprehension. In Proc. Fourth Workshop on Sch. Document Process.. 293–306. find
  137. Kayvan Kousha and Mike Thelwall. 2024. Artificial intelligence to support publishing and peer review: A summary and review. Learn. Publ. 37, 1 (2024), 4–12. find
  138. Asheesh Kumar, Tirthankar Ghosal, and Asif Ekbal. 2021. A Deep Neural Architecture for Decision-Aware Meta-Review Generation. In 2021 ACM/IEEE Jt. Conf. on Digit. Libr.. 222–225. find
  139. Shrinidhi Kumbhar, Venkatesh Mishra, Kevin Coutinho, Divij Handa, Ashif Iquebal, and Chitta Baral. 2025. Hypothesis generation for materials discovery and design using goal-driven and constraint-guided LLM agents. arXiv:2501.13299 find
  140. Nino Künzli, Anke Berger, Katarzyna Czabanowska, Raquel Lucas, Andrea Madarasova Geckova, Sarah Mantwill, and Olaf von Dem Knesebeck. 2022. «I Do Not Have Time» - Is This the End of Peer Review in Public Health Sciences? Public Health Rev. 43 (2022). find
  141. Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, et al. 2024. What Can Natural Language Processing Do for Peer Review? arXiv:2405.06563 find
  142. Ilia Kuznetsov, Jan Buchmann, Max Eichler, and Iryna Gurevych. 2022. Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review. Comput. Linguist. 48, 4 (12 2022), 949–986.
  143. Jon M. Laurent, Joseph D. Janizek, Michael Ruzo, Michaela M. Hinks, Michael J. Hammerling, Siddharth Narayanan, Manvitha Ponnapati, Andrew D White, and Samuel G. Rodriques. 2024. Lab-bench: Measuring capabilities of language models for biology research. arXiv:2407.10362 find
  144. Po-Shen Lee, Jevin D. West, and Bill Howe. 2016. Viziometrics: Analyzing Visual Information in the Scientific Literature. IEEE Trans. on Big Data 4 (2016), 117–129. find
  145. Yukyung Lee, Joonghoon Kim, Jaehee Kim, Hyowon Cho, Jaewook Kang, Pilsung Kang, and Najoung Kim. 2025. Checkeval: A reliable llm-as-a-judge framework for evaluating text generation using checklists. In Proc. 2025 Conf. on Empir. Methods Nat. Lang. Process.. 15782–15809.
  146. Christoph Leiter, Jonas Belouadi, Yanran Chen, Ran Zhang, Daniil Larionov, Aida Kostikova, and Steffen Eger. 2024. NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers? arXiv:2412.12121 — [arXiv:2412.12121](https://arxiv.org/abs/2412.12121) find
  147. Adrian Letchford, Helen Susannah Moat, and Tobias Preis. 2015. The Advantage of Short Paper Titles. Royal Soc. Open Sci. 2, 8 (2015). find
  148. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proc. NeurIPS, Vol. 33. 9459–9474.
  149. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, et al. 2025. From generation to judgment: Opportunities and challenges of LLM-as-a-judge. In Proc. 2025 Conf. on Empir. Methods Nat. Lang. Process.. 2757–2… find
  150. Da-Wei Li, Danqing Huang, Tingting Ma, and Chin-Yew Lin. 2021. Towards Topic-Aware Slide Generation For Academic Papers With Unsupervised Mutual Learning. In AAAI Conf. on Artif. Intell.. find
  151. Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models. In Proc. 2023 Conf. on Empir. Methods Nat. Lang. Process., Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). 6449–6464.
  152. Jiyi Li, Ayaka Sato, Kazuya Shimura, and Fumiyo Fukumoto. 2020. Multi-task Peer-Review Score Prediction. In Proc. First Workshop on Sch. Document Process.. 121–126. find
  153. Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024. Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models. In Proc. 62nd Annu. Meet. Assoc. for Comput. Linguist., Vol. 1. 14369–14387.
  154. Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al. 2024. Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents. arXiv:2410.13185 find
  155. Miao Li, Eduard Hovy, and Jey Lau. 2023. Summarizing Multiple Documents with Conversational Structure for Meta-Review Generation. In Find. Assoc. for Comput. Linguist. EMNLP 2023. 7089–7112. find
  156. Shujie Li, Liang Li, Ruiying Geng, Min Yang, Binhua Li, Guanghu Yuan, Wanwei He, Shao Yuan, Can Ma, Fei Huang, and Yongbin Li. 2024. Unifying Structured Data as Graph for Data-to-Text Pre-Training. Trans. Assoc. for Comput. Linguist. 12 (2024), 210–228. Manuscript submitted to ACM find
  157. Xiangci Li, Biswadip Mandal, and Jessica Ouyang. 2022. CORWA: A Citation-Oriented Related Work Annotation Dataset. In Proc. 2022 Conf. North Am. Chapter Assoc. for Comput. Linguist., Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). 5426–5440. find
  158. Xiangci Li and Jessica Ouyang. 2024. Related Work and Citation Text Generation: A Survey. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 13846–13864. find
  159. Xiangci Li and Jessica Ouyang. 2025. Explaining relationships among research papers. In Proc. 31st Int. Conf. on Comput. Linguist.. 1080–1105.
  160. Yutong Li, Lu Chen, Aiwei Liu, Kai Yu, and Lijie Wen. 2025. ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary. In Proc. 31st Int. Conf. on Comput. Linguist.. 3613–3630.
  161. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv:2308.03281
  162. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, et al. 2023. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv:2310.01783 find
  163. Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, et al. 2025. Quantifying large language model usage in scientific papers. Nat. Hum. Behav. (2025), 1–11. find
  164. Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D Manning, and James Y. Zou. 2024. Mapping the Increasing Use of LLMs in Scientific Papers. In Proc. COLM. find
  165. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summ. Branches Out. 74–81.
  166. Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. 2023. Automated scholarly paper review: Concepts, technologies, and challenges. Inf. Fusion 98 (2023), 101830. find
  167. Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. 2023. MOPRD: A multidisciplinary open peer review dataset. Neural Comput. Appl. 35, 34 (Sept. 2023), 24191–24206. find
  168. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. TruthfulQA: Measuring how models mimic human falsehoods. arXiv:2109.07958
  169. Cencen Liu, Yi Xu, Wen Yin, and Dezhang Zheng. 2023. Structure-aware Table-to-Text Generation with Prefix-tuning. In Proc. 2023 4th Int. Conf. on Control. Robotics Intell. Syst.. 135–140. find
  170. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation. In Proc. 2016 Conf. on Empir. Methods Nat. Lang. Process., Jian Su, Kev…
  171. Haokun Liu, Sicong Huang, Jingyu Hu, Yangqiaoyu Zhou, and Chenhao Tan. 2025. Hypobench: Towards systematic and principled benchmarking for hypothesis generation. arXiv:2504.11524 — [arXiv:2504.11524](https://arxiv.org/abs/2504.11524) find
  172. Quanliang Liu, Maciej P Polak, So Yeon Kim, MD Shuvo, Hrishikesh Shridhar Deodhar, Jeongsoo Han, Dane Morgan, and Hyunseok Oh. 2024. Beyond designer’s knowledge: Generating materials design hypotheses via large language models. arXiv:2409.06756 — [arXiv:2409.06756](https://arxiv.org/abs/2409.06756) find
  173. Ryan Liu and Nihar B. Shah. 2023. ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing. arXiv:arXiv:2306.00622 find
  174. Sizhe Liu, Yizhou Lu, Siyu Chen, Xiyang Hu, Jieyu Zhao, Tianfan Fu, and Yue Zhao. 2024. DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration. arXiv:2411.15692 find
  175. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment. In Proc. 2023 Conf. on Empir. Methods Nat. Lang. Process., Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). 2511–2522.
  176. Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Dan S Weld. 2019. S2ORC: The semantic scholar open research corpus. arXiv:1911.02782
  177. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292
  178. Yuyu Luo, Jiawei Tang, and Guoliang Li. 2021. nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task. arXiv:2112.12926 — [arXiv:2112.12926](https://arxiv.org/abs/2112.12926) find
  179. Ziming Luo, Zonglin Yang, Zexin Xu, Wei Yang, and Xinya Du. 2025. LLM4SR: A Survey on Large Language Models for Scientific Research. arXiv:2501.04306 find
  180. Kangyong Ma. 2025. AI agents in chemical research: GVIM – An intelligent research assistant system. Digit. Discov. (2025). find
  181. Himanshu Maheshwari, Sambaran Bandyopadhyay, Aparna Garimella, and Anandhavelu Natarajan. 2024. Presentations are not always linear! GNN meets LLM for Text Document-to-Presentation Transformation with Attribution. In Find. Assoc. for Comput. Linguist. EMNLP 2024. 15948–15962. find
  182. Anna Martin-Boyle, Aahan Tyagi, Marti A Hearst, and Dongyeop Kang. 2024. Shallow synthesis of knowledge in GPT-generated texts: A case study in automatic related work composition. arXiv:2402.12255
  183. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. In Find. Assoc. for Comput. Linguist. ACL 2022, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). 2263–2279.
  184. Prakhar Mishra, Chaitali Diwan, Srinath Srinivasa, and Gopalakrishnan Srinivasaraghavan. 2021. Automatic title generation for text with pre-trained transformer language model. In 2021 IEEE 15th Int. Conf. on Semantic Comput.. IEEE, 17–24. find
  185. Ishani Mondal, Zongxia Li, Yufang Hou, Anandhavelu Natarajan, Aparna Garimella, and Jordan Lee Boyd-Graber. 2024. SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinement. In Find. Assoc. for Comput. Linguist. EMNLP 2024. 13342–13375.… find
  186. Ishani Mondal, Shwetha S, Anandhavelu Natarajan, Aparna Garimella, Sambaran Bandyopadhyay, and Jordan Boyd-Graber. 2024. Presentations by the Humans and For the Humans: Harnessing LLMs for Generating Persona-Aware Slides from Documents. In Proc. EACL. 2664–2684. find
  187. Nafise Moosavi, Andreas Rücklé, Dan Roth, and Iryna Gurevych. 2021. SciGen: a Dataset for Reasoning-Aware Text Generation from Scientific Tables. In Proc. Neural Inf. Process. Syst. Track on Datasets Benchmarks, Vol. 1. find
  188. Rajiv Movva, Kenny Peng, Nikhil Garg, Jon Kleinberg, and Emma Pierson. 2025. Sparse autoencoders for hypothesis generation. arXiv:2502.04382 — [arXiv:2502.04382](https://arxiv.org/abs/2502.04382) find
  189. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive Text Embedding Benchmark. In Proc. 17th Conf. Eur. Chapter Assoc. for Comput. Linguist.. 2014–2037.
  190. Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, Dieuwke Hupkes, Ricardo Silveira Cabral, Tatiana Shavrina, Jakob Foerster, Yoram Bachrach, William Yang Wang, and Roberta Raileanu. 20… find
  191. Benjamin Newman, Yoonjoo Lee, Aakanksha Naik, Pao Siangliulue, Raymond Fok, Juho Kim, Daniel S Weld, Joseph Chee Chang, and Kyle Lo. 2024. ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 9612–9631. find
  192. Yuansheng Ni, Ping Nie, Kai Zou, Xiang Yue, and Wenhu Chen. 2025. VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation. arXiv:2506.03930 find
  193. Allard Oelen, Mohamad Yaser Jaradeh, and Sören Auer. 2025. Introducing ORKG ASK: An AI-Driven Scholarly Literature Search and Exploration System Taking a Neuro-Symbolic Approach. In Int. Conf. on Web Eng.. 11–25. find
  194. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proc. 40th Annu. Meet. Assoc. for Comput. Linguist., Pierre Isabelle, Eugene Charniak, and Dekang Lin (Eds.). 311–318.
  195. Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proc. 2020 Conf. on Empir. Methods Nat. Lang. Process., Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). 1173–1186.
  196. Yang Jeong Park, Daniel Kaplan, Zhichu Ren, Chia-Wei Hsu, Changhao Li, Haowei Xu, Sipei Li, and Ju Li. 2024. Can ChatGPT be used to generate scientific hypotheses? J. Materiomics 10, 3 (2024), 578–584. find
  197. Guy Paré, Marie-Claude Trudel, Mirou Jaana, and Spyros Kitsiou. 2015. Synthesizing Information Systems Knowledge: a Typology of Literature Reviews. Inf. & Manag. 52 (2015), 183–199. find
  198. Dominic Petrak, Nafise Sadat Moosavi, and Iryna Gurevych. 2023. Arithmetic-Based Pretraining Improving Numeracy of Pretrained Language Models. In Proc. 12th Jt. Conf. on Lexical Comput. Semant., Alexis Palmer and Jose Camacho-collados (Eds.). 477–493. find
  199. Nicky Phillips. 2017. Online software spots genetic errors in cancer papers. Nature 551, 7681 (2017). find
  200. Barbara Plank and Reinard van Dalen. 2019. CiteTracked: A longitudinal dataset of peer reviews and citations. In Proc. 4th Jt. Workshop on Bibliometric-enhanced Inf. Retr. Nat. Lang. Process. for Digit. Libr.. 116–122. find
  201. Aniket Pramanick, Yufang Hou, Saif M. Mohammad, and Iryna Gurevych. 2024. The Nature of NLP: Analyzing Contributions in NLP Papers. arXiv:2409.19505
  202. Aniket Pramanick, Yufang Hou, Saif M. Mohammad, and Iryna Gurevych. 2024. Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Science. arXiv:2409.19508 — [arXiv:2409.19508](https://arxiv.org/abs/2409.19508) find
  203. Shraman Pramanick, Rama Chellappa, and Subhashini Venugopalan. 2024. SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers. In Adv. Neural Inf. Process. Syst. 38, Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (Eds.). find
  204. Kevin Pu, KJ Feng, Tovi Grossman, Tom Hope, Bhavana Dalvi Mishra, Matt Latzke, Jonathan Bragg, Joseph Chee Chang, and Pao Siangliulue. 2024. IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback. arXiv:2410.04025 — [arXiv:2410.04025](https://arxiv.org/abs/2410.04025) find
  205. Zhongji Pu, Chun-Lin Shi, Che Ok Jeon, Jingyuan Fu, Shuang-Jiang Liu, Canhui Lan, Yanlai Yao, Yong-Xin Liu, and Baolei Jia. 2024. ChatGPT and generative AI are revolutionizing the scientific community: A Janus-faced conundrum. iMeta 3, 2 (2024), e178. find
  206. Sukannya Purkayastha, Nils Dycke, Anne Lauscher, and Iryna Gurevych. 2025. Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue. arXiv:2508.05283
  207. Sukannya Purkayastha, Anne Lauscher, and Iryna Gurevych. 2023. Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttals. In Proc. 2023 Conf. on Empir. Methods Nat. Lang. Process.. 14479–14495. find
  208. Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen, and Bowen Zhou. 2023. Large language models are zero shot hypothesis proposers. arXiv:2311.05965 find
  209. Biqing Qi, Kaiyan Zhang, Kai Tian, Haoxiang Li, Zhang-Ren Chen, Sihang Zeng, Ermo Hua, Hu Jinfang, and Bowen Zhou. 2024. Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation. arXiv:2407.08940
  210. Yuting Qiang, Yanwei Fu, Yanwen Guo, Zhi-Hua Zhou, and Leonid Sigal. 2016. Learning to Generate Posters of Scientific Papers. In AAAI Conf. on Artif. Intell.. find
  211. Marissa Radensky, Simra Shahid, Raymond Fok, Pao Siangliulue, Tom Hope, and Daniel S Weld. 2024. Scideator: Human-LLM Scientific Idea Generation Grounded in Research-Paper Facet Recombination. arXiv:2409.14634 Manuscript submitted to ACM — [arXiv:2409.14634](https://arxiv.org/abs/2409.14634) find
  212. Raian Rahman, Rizvi Hasan, Abdullah Al Farhad, Md Tahmid Rahman Laskar, Md. Hamjajul Ashmafee, and Abu Raihan Mostofa Kamal. 2023. ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries. arXiv:2304.13620
  213. Pritika Ramu, Aparna Garimella, and Sambaran Bandyopadhyay. 2024. Is This a Bad Table? A Closer Look at the Evaluation of Table Generation from Text. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 22206–22216. find
  214. Chandan K Reddy and Parshin Shojaee. 2024. Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges. arXiv:2412.11427 — [arXiv:2412.11427](https://arxiv.org/abs/2412.11427) find
  215. Isabelle Régner, Catherine Thinus-Blanc, Agnès Netter, Toni Schmader, and Pascal Huguet. 2019. Committees with implicit biases promote fewer women when they do not believe gender bias exists. Nat. Hum. Behav. 3, 11 (2019), 1171–1179. find
  216. Georg Rehm, Stefan Dietze, Sonja Schimmler, and Frank Krüger. 2024. Natural Scientific Language Processing and Research Knowledge Graphs: First International Workshop, NSLP 2024. Springer. find
  217. Zachary Robertson. 2023. GPT-4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study. arXiv:2307.05492 — [arXiv:2307.05492](https://arxiv.org/abs/2307.05492) find
  218. Helmi Ben Saad, Ismail Dergaa, Hatem Ghouili, Halil İbrahim Ceylan, Karim Chamari, and Wissem Dhahbi. 2025. The Assisted Technology Dilemma: A Reflection on AI Chatbots Use and Risks While Reshaping the Peer Review Process in Scientific Research. AI Soc. 40, 7 (2025), 5649–5656. find
  219. Furkan Şahinuç, Subhabrata Dutta, and Iryna Gurevych. 2025. Expert Preference-based Evaluation of Automated Related Work Generation. arXiv:2508.07955 — [arXiv:2508.07955](https://arxiv.org/abs/2508.07955) find
  220. Furkan Şahinuç, Ilia Kuznetsov, Yufang Hou, and Iryna Gurevych. 2024. Systematic Task Exploration with LLMs: A Study in Citation Text Generation. In Proc. 62nd Annu. Meet. Assoc. for Comput. Linguist., Vol. 1. 4832–4855. find
  221. Furkan Şahinuç, Thy Thy Tran, Yulia Grishina, Yufang Hou, Bei Chen, and Iryna Gurevych. 2024. Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboards. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 7963–7977. find
  222. Michele Salvagno, Fabio Silvio Taccone, and Alberto Giovanni Gerli. 2023. Can artificial intelligence help for scientific writing? Crit. care 27, 1 (2023), 75. find
  223. Shubhra Kanti Karmaker Santu, Sanjeev Kumar Sinha, Naman Bansal, Alex Knipper, Souvika Sarkar, John Salvador, Yash Mahajan, Sri Guttikonda, Mousumi Akter, Matthew Freestone, and Matthew C. Williams Jr. 2024. Prompting LLMs to Compose Meta-Review Drafts from Peer-Review Narratives of Scholarly Manusc… — [arXiv:2402.15589](https://arxiv.org/abs/2402.15589) find
  224. Laurie A. Schintler, Connie L. McNeely, and James Witte. 2023. A Critical Examination of the Ethics of AI-Mediated Peer Review. arXiv:2309.12356 — [arXiv:2309.12356](https://arxiv.org/abs/2309.12356) find
  225. Daniel Schlagwein and Leslie Willcocks. 2023. ‘ChatGPT et al.’: The ethics of using (generative) artificial intelligence in research and science. J. Inf. Technol. 38, 3 (2023), 232–238. find
  226. Michael Schlichtkrull, Nedjma Ousidhoum, and Andreas Vlachos. 2023. The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Who. In Find. Assoc. for Comput. Linguist. EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). 8618–8642. find
  227. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using LLM agents as research assistants. arXiv:2501.04227
  228. Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. 2024. AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments. arXiv:2405.07960 — [arXiv:2405.07960](https://arxiv.org/abs/2405.07960) find
  229. Dominik Schmidt, Zhengyao Jiang, and Yuxiang Wu. 2024. Introducing Weco AIDE. https://www.weco.ai/blog/technical-report. ↗
  230. SciScore. 2024. The best methods review tool for scientific research. https://sciscore.com/. ↗
  231. Paul Sebo, Bing Nie, and Ting Wang. 2025. Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titles. F1000Research 14 (2025), 1470. find
  232. Chenhui Shen, Liying Cheng, Ran Zhou, Lidong Bing, Yang You, and Luo Si. 2022. MReD: A Meta-Review Dataset for Structure-Controllable Text Generation. In Find. Assoc. for Comput. Linguist. ACL 2022. 2521–2535. find
  233. Pranav Shetty, Arunkumar Chitteth Rajan, Chris Kuenneth, Sonakshi Gupta, Lakshmi Prerana Panchumarti, Lauren Holm, Chao Zhang, and Rampi Ramprasad. 2023. A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing. npj Comput. Mater. 9, 1… find
  234. Chufan Shi, Cheng Yang, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. 2024. ChartMimic: Evaluating LMM’s Cross-Modal Reasoning Capability via Chart-to-Code Generation. arXiv:2406.09961 — [arXiv:2406.09961](https://arxiv.org/abs/2406.09961) find
  235. Haoxiang Shi, Jiaan Wang, Jiarong Xu, Cen Wang, and Tetsuya Sakai. 2024. CT-Eval: Benchmarking Chinese Text-to-Table Performance in Large Language Models. arXiv:2405.12174 — [arXiv:2405.12174](https://arxiv.org/abs/2405.12174) find
  236. Chenglei Si, Tatsunori Hashimoto, and Diyi Yang. 2025. The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas. arXiv:2506.20803 find
  237. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. 2024. Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. arXiv:2409.04109
  238. Shruti Singh, Mayank Singh, and Pawan Goyal. 2021. COMPARE: A Taxonomy and Dataset of Comparison Discussions in Peer Reviews. arXiv:2108.04366 — [arXiv:2108.04366](https://arxiv.org/abs/2108.04366) find
  239. Henry Small. 1973. Co-citation in the scientific literature: A new measure of the relationship between two documents. J. Am. Soc. for Inf. Sci. 24, 4 (1973), 265–269. Manuscript submitted to ACM find
  240. Wael Soliman and Mikko Siponen. 2022. What Do We Really Mean by Rigor in Information Systems Research?. In Proc. 55th Hawaii Int. Conf. on Syst. Sci.. find
  241. Karen Spärck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. J. Documentation 28, 1 (1972), 11–21. find
  242. M. Sravanthi, C. Ravindranath Chowdary, and P. Sreenivasa Kumar. 2009. SlidesGen: Automatic Generation of Presentation Slides for a Technical Paper Using Summarization. In The Fla. AI Res. Soc.. find
  243. Lukas Stappen, Georgios Rizos, Madina Hasan, Thomas Hain, and Björn W Schuller. 2020. Uncertainty-aware machine support for paper reviewing on the Interspeech 2019 submission corpus. In Proc. Interspeech 2020. 1808–1812. find
  244. Andrew Starnes and Clayton Webster. 2024. Mamba for Scalable and Efficient Personalized Recommendations. arXiv:2409.17165 — [arXiv:2409.17165](https://arxiv.org/abs/2409.17165) find
  245. Moritz Staudinger, Wojciech Kusa, Florina Piroi, and Allan Hanbury. 2024. An Analysis of Tasks and Datasets in Peer Reviewing. In Proc. Fourth Workshop on Sch. Document Process.. 257–268. find
  246. Dana Strauss, Sophia Gran-Ruaz, Muna Osman, Monnica T. Williams, and Sonya C. Faber. 2023. Racism and censorship in the editorial and peer review process. Front. Psychol. 14 (2023). find
  247. Haoyang Su, Renqi Chen, Shixiang Tang, Xinzhe Zheng, Jingzhe Li, Zhenfei Yin, Wanli Ouyang, and Nanqing Dong. 2024. Two Heads Are Better Than One: A Multi-Agent System Has the Potential to Improve Scientific Idea Generation. arXiv:2410.09403 — [arXiv:2410.09403](https://arxiv.org/abs/2410.09403) find
  248. Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura, and Hiroya Takamura. 2021. Towards Table-to-Text Generation with Numerical Reasoning. In Proc. 59th Annu. Meet. Assoc. for Comput. Linguist. 11th Int. Jt. Conf. on Nat. Lang. Process., Chengqing Zong, Fei Xia, Wenjie Li, a… find
  249. Edward Sun, Yufang Hou, Dakuo Wang, Yunfeng Zhang, and Nancy X. R. Wang. 2021. D2S: Document-to-Slide Generation Via Query-Based Text Summarization. In Proc. NAACL. 1405–1418. find
  250. Kexuan Sun, Zhiqiang Qiu, Abel Salinas, Yuzhong Huang, Dong-Ho Lee, Daniel Benjamin, Fred Morstatter, Xiang Ren, Kristina Lerman, and Jay Pujara. 2022. Assessing Scientific Research Papers with Knowledge Graphs. In SIGIR ’22: The 45th Int. ACM SIGIR Conf. on Res. Dev. Inf. Retr., Enrique Amigó, Pabl… find
  251. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, et al. 2024. TrustLLM: Trustworthiness in large language models. arXiv:2401.05561
  252. Anirudh Sundar, Christopher Richardson, and Larry Heck. 2024. gTBLS: Generating Tables from Text by Conditional Question Answering. arXiv:2403.14457 — [arXiv:2403.14457](https://arxiv.org/abs/2403.14457) find
  253. Jaroslaw Szumega, Lamine Bougueroua, Blerina Gkotse, Pierre Jouvelot, and Federico Ravotti. 2023. The Open Review-Based (ORB) dataset: Towards Automatic Assessment of Scientific Papers and Experiment Proposals in High-Energy Physics. arXiv:2312.04576 — [arXiv:2312.04576](https://arxiv.org/abs/2312.04576) find
  254. Neset Tan, Trung Nguyen, Josh Bensemann, Alex Peng, Qiming Bao, Yang Chen, Mark Gahegan, and Michael Witbrock. 2023. Multi2Claim: Generating Scientific Claims from Multi-Choice Questions for Scientific Fact-Checking. In Proc. 17th Conf. Eur. Chapter Assoc. for Comput. Linguist.. 2652–2664. find
  255. Liyan Tang, Yifan Peng, Yanshan Wang, Ying Ding, Greg Durrett, and Justin Rousseau. 2023. Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses. In Find. Assoc. for Comput. Linguist. ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 12532–12555. find
  256. Song Tong, Kai Mao, Zhen Huang, Yukun Zhao, and Kaiping Peng. 2024. Automating Psychological Hypothesis Generation with AI: Large Language Models Meet Causal Graph. arXiv:2402.14424 — [arXiv:2402.14424](https://arxiv.org/abs/2402.14424) find
  257. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288
  258. Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. 2024. AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML. arXiv:2410.02958 — [arXiv:2410.02958](https://arxiv.org/abs/2410.02958) find
  259. Yun-Da Tsai, Yu-Che Tsai, Bo-Wei Huang, Chun-Pai Yang, and Shou-De Lin. 2023. AutoML-GPT: Large language model for AutoML. arXiv:2309.01125 — [arXiv:2309.01125](https://arxiv.org/abs/2309.01125) find
  260. Juraj Vladika and Florian Matthes. 2023. Scientific Fact-Checking: A Survey of Resources and Approaches. In Find. Assoc. for Comput. Linguist. ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 6215–6230. find
  261. Henrik Voigt, Kai Lawonn, and Sina Zarrieß. 2024. Plots Made Quickly: An Efficient Approach for Generating Visualizations from Natural Language Queries. In Proc. 2024 Jt. Int. Conf. on Comput. Linguist. Lang. Resour. Eval.. 12787–12793. find
  262. Dario von Wedel, Rico A. Schmitt, Moritz Thiele, Raphael Leuner, Denys Shay, Simone Redaelli, and Maximilian S. Schaefer. 2024. Affiliation Bias in Peer Review of Abstracts by a Large Language Model. JAMA 331, 3 (01 2024), 252–253. find
  263. Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, et al. 2023. FreshLLMs: Refreshing large language models with search engine augmentation. arXiv:2310.03214
  264. David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, and Hannaneh Hajishirzi. 2022. SciFact-Open: Towards open-domain scientific claim verification. In Find. Assoc. for Comput. Linguist. EMNLP 2022, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). 4719–4734.
  265. William H. Walters and Esther Isabelle Wilder. 2023. Fabrication and errors in the bibliographic citations generated by ChatGPT. Sci. Reports 13, 1 (2023), 14045. find
  266. Fei Wang, Zhewei Xu, Pedro Szekely, and Muhao Chen. 2022. Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning. In Proc. 2022 Conf. North Am. Chapter Assoc. for Comput. Linguist., Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). 5037… find
  267. Fuchun Wang, Xian Zhou, Wenpeng Hu, Zhunchen Luo, Wei Luo, and Xiaoying Bai. 2024. LLM Assists Hypothesis Generation and Testing for Deliberative Questions. In CCF Int. Conf. on Nat. Lang. Process. Chin. Comput.. Springer, 424–436. find
  268. Hao Wang and Wu-Jun Li. 2014. Relational collaborative topic regression for recommender systems. IEEE Trans. on Knowl. Data Eng. 27, 5 (2014), 1343–1355. find
  269. Pancheng Wang, Shasha Li, Haifang Zhou, Jintao Tang, and Ting Wang. 2019. ToC-RWG: Explore the Combination of Topic Model and Citation Information for Automatic Related Work Generation. IEEE Access 8 (2019), 13043–13055. find
  270. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. 2024. Scimon: Scientific inspiration machines optimized for novelty. In Proc. 62nd Annu. Meet. Assoc. for Comput. Linguist.. 279–299. find
  271. Qingyun Wang, Lifu Huang, Zhiying Jiang, Kevin Knight, Heng Ji, Mohit Bansal, and Yi Luan. 2019. PaperRobot: Incremental Draft Generation of Scientific Ideas. In Proc. 57th Annu. Meet. Assoc. for Comput. Linguist., Anna Korhonen, David Traum, and Lluís Màrquez (Eds.). 1980–1991. find
  272. Qingyun Wang, Qi Zeng, Lifu Huang, Kevin Knight, Heng Ji, and Nazneen Fatema Rajani. 2020. ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis. In Proc. 13th Int. Conf. on Nat. Lang. Gener.. 384–397. find
  273. Sida Wang, Xiaojun Wan, and Shikang Du. 2017. Phrase-Based Presentation Slides Generation for Academic Papers. In AAAI Conf. on Artif. Intell.. find
  274. Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. 2024. OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv:2407.16741
  275. Yubo Wang, Xueguang Ma, Ping Nie, Huaye Zeng, Zhiheng Lyu, Yuxuan Zhang, Benjamin Schneider, Yi Lu, Xiang Yue, and Wenhu Chen. 2025. ScholarCopilot: Training large language models for academic writing with accurate citations. arXiv:2504.00824 — [arXiv:2504.00824](https://arxiv.org/abs/2504.00824) find
  276. Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, and Danqi Chen. 2024. CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs. In Adv. Neural Inf. Process. Syst. 38, Amir… find
  277. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does LLM safety training fail? Adv. Neural Inf. Process. Syst. 36 (2023), 80079–80110.
  278. Jingxuan Wei, Cheng Tan, Qi Chen, Gaowei Wu, Siyuan Li, Zhangyang Gao, Linzhuang Sun, Bihui Yu, and Ruifeng Guo. 2025. From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing. In Proc. Comput. Vis. Pattern Recognit. Conf.. 13315–13325. find
  279. Ann C. Weller. 2001. Editorial Peer Review: Its Strengths and Weaknesses. American Society for Information Science and Technology. find
  280. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. 2025. CycleResearcher: Improving Automated Research via Automated Review. In The Thirteen. Int. Conf. on Learn. Represent.. find
  281. Leslie P. Willcocks, John Hindle, Matt Stanton, and John Smith. 2023. Maximizing Value with Automation and Transformation: A Realist’s Guide. Palgrave Macmillan. find
  282. Siwei Wu, Yizhi Li, Xingwei Qu, Rishi Ravikumar, Yucheng Li, Tyler Loakman Shanghaoran Quan Xiaoyong Wei, Riza Batista-Navarro, and Chenghua Lin. 2025. LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm. arXiv:2502.19103 — [arXiv:2502.19103](https://arxiv.org/abs/2502.19103) find
  283. Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, and Chenghua Lin. 2024. SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval. In Find. Assoc. for Comput. Linguist. ACL 2024.… find
  284. Yuhao Wu, Yushi Bai, Zhiqiang Hu, Roy Ka-Wei Lee, and Juanzi Li. 2025. LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning. arXiv:2506.18841 — [arXiv:2506.18841](https://arxiv.org/abs/2506.18841) find
  285. Guangzhi Xiong, Eric Xie, Amir Hassan Shariatmadari, Sikun Guo, Stefan Bekiranov, and Aidong Zhang. 2024. Improving Scientific Hypothesis Generation with Knowledge Grounded Large Language Models. arXiv:2411.02382
  286. Peixin Xu, Yujuan Ding, and Wenqi Fan. 2024. ChartAdapter: Large Vision-Language Model for Chart Summarization. arXiv:2412.20715 — [arXiv:2412.20715](https://arxiv.org/abs/2412.20715) find
  287. Ruoxi Xu, Yingfei Sun, Mengjie Ren, Shiguang Guo, Ruotong Pan, Hongyu Lin, Le Sun, and Xianpei Han. 2024. AI for social science and social science of AI: A survey. Inf. Process. & Manag. 61, 3 (2024), 103665. find
  288. Sheng Xu and Xiaojun Wan. 2022. PosterBot: A System for Generating Posters of Scientific Papers with Neural Models. In AAAI Conf. on Artif. Intell.. find
  289. Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. 2025. The AI Scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv:2504.08066
  290. Hidekazu Yanagimoto, Iroha Kisaku, and Kiyota Hashimoto. 2024. Table-to-Text Using Pre-trained Large Language Model and LoRA. In 2024 16th IIAI Int. Congr. on Adv. Appl. Informatics. 91–96. find
  291. Bohao Yang, Yingji Zhang, Dong Liu, André Freitas, and Chenghua Lin. 2025. Does table source matter? Benchmarking and improving multimodal scientific table understanding and reasoning. arXiv:2501.13042 — [arXiv:2501.13042](https://arxiv.org/abs/2501.13042) find
  292. Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. 2023. Large language models for automated open-domain scientific hypotheses discovery. arXiv:2309.02726 find
  293. Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria, and Dongzhan Zhou. 2024. MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses. arXiv:2410.07076 find
  294. Michihiro Yasunaga, Jungo Kasai, Rui Zhang, Alexander R. Fabbri, Irene Li, Dan Friedman, and Dragomir R. Radev. 2019. ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks. In The Thirty-Third AAAI Conf. on Artif. Manuscript submitte… find
  295. Deming Ye, Yankai Lin, Peng Li, and Maosong Sun. 2021. Packed levitated marker for entity and relation extraction. arXiv:2109.06067
  296. Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed Chi. 2019. Sampling-bias-corrected neural modeling for large corpus item recommendations. In Proc. 13th ACM Conf. on Recomm. Syst.. 269–277. find
  297. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do Large Language Models Know What They Don’t Know?. In Find. Assoc. for Comput. Linguist. ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 8653–8665. find
  298. Larry D. Yore, Brian M. Hand, and Marilyn K. Florence. 2004. Scientists’ Views of Science, Models of Writing, and Science Writing Practices. J. Res. Sci. Teach. 41, 4 (2004), 338–369. find
  299. Yantao Yu, Weipeng Wang, Zhoutian Feng, and Daiyue Xue. 2021. A dual augmented two-tower model for online large-scale recommendation. In Proc. DLP-KDD. find
  300. Weizhe Yuan, Pengfei Liu, and Graham Neubig. 2022. Can We Automate Scientific Reviewing? J. Artif. Intell. Res. 75 (2022). find
  301. Abhay Zala, Han Lin, Jaemin Cho, and Mohit Bansal. 2024. DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning. In COLM. find
  302. Qi Zeng, Mankeerat Sidhu, Hou Pong Chan, Lu Wang, and Heng Ji. 2023. Meta-review Generation with Checklist-guided Iterative Introspection. arXiv:2305.14647 — [arXiv:2305.14647](https://arxiv.org/abs/2305.14647) find
  303. Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, and Juanzi Li. 2024. LongReward: Improving Long-context Large Language Models with AI Feedback. arXiv:2410.21252
  304. Leixin Zhang, Steffen Eger, Yinjie Cheng, Weihe Zhai, Jonas Belouadi, Christoph Leiter, Simone Paolo Ponzetto, Fahimeh Moafian, and Zhixue Zhao. 2024. ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation? arXiv:2412.02368 — [arXiv:2412.02368](https://arxiv.org/abs/2412.02368) find
  305. Lei Zhang, Yuge Zhang, Kan Ren, Dongsheng Li, and Yuqing Yang. 2023. Mlcopilot: Unleashing the power of large language models in solving machine learning tasks. arXiv:2304.14979
  306. Qiang Zhang, Wanyi Chen, Ming Qin, Yuhao Wang, Zhongji Pu, Keyan Ding, Yuyue Liu, Qunfeng Zhang, Dongfang Li, Xinjia Li, et al. 2025. Integrating protein language models and automatic biofoundry for enhanced protein evolution. Nat. Commun. 16, 1 (2025), 1553. find
  307. Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yu-Ching Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, and etc. 2023. Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems. arXiv:2307.08423 find
  308. Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang, and Jiawei Han. 2024. A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery. In Proc. 2024 Conf. on Empir. Methods Nat. Lang. Process.. 8783–8817. find
  309. Yunyi Zhang, Ruozhen Yang, Siqi Jiao, SeongKu Kang, and Jiawei Han. 2025. Scientific Paper Retrieval with LLM-Guided Semantic-Based Ranking. In Find. Assoc. for Comput. Linguist. EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). 2049–2060. find
  310. Wei Zhao, Jennifer D’Souza, Steffen Eger, Anne Lauscher, Yufang Hou, Nafise Sadat Moosavi, Tristan Miller, and Chenghua Lin (Eds.). 2025. Proceedings of the First Workshop on Human–LLM Collaboration for Ethical and Responsible Science Production (SciProdLLM). find
  311. Xueliang Zhao, Tingchen Fu, Lemao Liu, Lingpeng Kong, Shuming Shi, and Rui Yan. 2023. SORTIE: Dependency-Aware Symbolic Reasoning for Logical Data-to-text Generation. In Find. Assoc. for Comput. Linguist. ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 11247–11266. find
  312. Yilun Zhao, Zhenting Qi, Linyong Nan, Lorenzo Jaime Flores, and Dragomir Radev. 2023. LoFT: Enhancing Faithfulness and Diversity for Table-to-Text Generation via Logic Form Control. In Proc. 17th Conf. Eur. Chapter Assoc. for Comput. Linguist., Andreas Vlachos and Isabelle Augenstein (Eds.). 554–561… find
  313. Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. 2024. OpenCodeInterpreter: Integrating code generation with execution and refinement. arXiv:2402.14658 find
  314. Zhiling Zheng, Oufan Zhang, Christian Borgs, Jennifer T Chayes, and Omar M Yaghi. 2023. ChatGPT chemistry assistant for text mining and the prediction of MOF synthesis. J. Am. Chem. Soc. 145, 32 (2023), 18048–18062. find
  315. Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. 2024. Hypothesis Generation with Large Language Models. arXiv:2404.04326 find
  316. Yujun Zhou, Jingdong Yang, Yue Huang, Kehan Guo, Zoe Emory, Bikram Ghosh, Amita Bedar, Sujay Shekar, Zhenwen Liang, Pin-Yu Chen, et al. 2026. Benchmarking large language models on safety risks in scientific laboratories. Nat. Mach. Intell. (2026), 1–12. find
  317. Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. 2024. Agent-as-a-Judge: Evaluate Agents with Agents. arXiv:2410.10934 Manuscript submi… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/eger-2025-transforming-science-large-language-models/