Evaluation of OpenAI o1: Opportunities and Challenges of AGI
Paper · 2024 · Tianyang Zhong, Zhengliang Liu, Yi Pan et al.
Summary
The paper examines how close OpenAI's o1-preview comes to general-purpose intelligence. o1-preview is a model trained with reinforcement learning to work through chain-of-thought reasoning before it answers. A large multi-institution team tested it on 27 task categories, mostly built from public datasets. These span programming, mathematics and statistics, and medicine (radiology reports, EHR diagnosis, anonymization of clinical text). They also include natural language inference, low-resource translation, chip design, robot command planning, 3D layout, quantitative investing, education, social media and sentiment analysis, and anthropology and geology. The collection is presented as AGI-Benchmark 1.0. Evaluation relies heavily on case studies that compare the model's outputs with reference answers or with other models. According to the authors, o1-preview often reaches or exceeds human-level results, particularly where tasks require multi-step reasoning and combining knowledge across fields. It still makes mistakes on some simple problems and on certain very specialized concepts. The work matters because it gives an early, broad picture of where reasoning-focused LLMs perform well. It also identifies remaining needs: multimodal integration, domain-specific validation and ethical safeguards.
Key points
- On complex competitive programming problems, o1-preview had a reported success rate of 83.3%.
- It scored 100% on high-school-level math reasoning tasks and gave step-by-step solutions.
- Its radiology report generation was rated better than the other evaluated models.
- In chip design, it beat specialized models on EDA script generation and bug summary/analysis.
- In the content summarization cases (one-sentence summaries of news articles), its outputs were judged only partly or fully consistent with the reference answers.
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works
🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works
Cite
Cite as Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Zeyu Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, Chao Cao, Hanqi Jiang, Hanxu Chen, Yiwei Li, Junhao Chen, Huawen Hu, Yiheng Liu, Huaqin Zhao, Shaochen Xu, Haixing Dai, Lin Zhao, Ruidong Zhang, Wei Zhao, Zhenyuan Yang, Jingyuan Chen, Peilong Wang, Wei Ruan, Hui Wang, Huan Zhao, Jing Zhang, Yiming Ren, Shihuan Qin, Tong Chen, Jiaxi Li, Arif Hassan Zidan, Afrar Jahin, Minheng Chen, Sichen Xia, Jason Holmes, Yan Zhuang, Jiaqi Wang, Bochen Xu, Weiran Xia, Jichao Yu, Kaibo Tang, Yaxuan Yang, Bolun Sun, Lifeng Chen, Tao Yang, Guoyu Lu, Xianqiao Wang, Lilong Chai, He Li, Jin Lu, Xin Zhang, Bao Ge, Xintao Hu, Lian Zhang, Hua Zhou, Lu Zhang, Shu Zhang, Zhen Xiang, Yudan Ren, Jun Liu, Xi Jiang, Yu Bao, Wei Zhang, Xiang Li, Gang Li, Wei Liu, Dinggang Shen, Andrea Sikora, Xiaoming Zhai, Dajiang Zhu, Tuo Zhang and Tianming Liu. 2024. Evaluation of OpenAI o1: Opportunities and Challenges of AGI. arXiv:2409.18486.
@misc{zhong2024evaluation,
title = {{Evaluation of OpenAI o1: Opportunities and Challenges of AGI}},
author = {Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Zeyu Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, Chao Cao, Hanqi Jiang, Hanxu Chen, Yiwei Li, Junhao Chen, Huawen Hu, Yiheng Liu, Huaqin Zhao, Shaochen Xu, Haixing Dai, Lin Zhao, Ruidong Zhang, Wei Zhao, Zhenyuan Yang, Jingyuan Chen, Peilong Wang, Wei Ruan, Hui Wang, Huan Zhao, Jing Zhang, Yiming Ren, Shihuan Qin, Tong Chen, Jiaxi Li, Arif Hassan Zidan, Afrar Jahin, Minheng Chen, Sichen Xia, Jason Holmes, Yan Zhuang, Jiaqi Wang, Bochen Xu, Weiran Xia, Jichao Yu, Kaibo Tang, Yaxuan Yang, Bolun Sun, Lifeng Chen, Tao Yang, Guoyu Lu, Xianqiao Wang, Lilong Chai, He Li, Jin Lu, Xin Zhang, Bao Ge, Xintao Hu, Lian Zhang, Hua Zhou, Lu Zhang, Shu Zhang, Zhen Xiang, Yudan Ren, Jun Liu, Xi Jiang, Yu Bao, Wei Zhang, Xiang Li, Gang Li, Wei Liu, Dinggang Shen, Andrea Sikora, Xiaoming Zhai, Dajiang Zhu, Tuo Zhang and Tianming Liu},
year = {2024},
doi = {10.48550/arxiv.2409.18486},
eprint = {2409.18486},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.2409.18486}
}References
- 1988 IMO Problems/Problem 6. https://artofproblemsolving.com/wiki/index.php? title=1988_IMO_Problems/Problem_6. [Online; accessed 19-Sep-2024]. Art of Problem Solving, 1988. ↗
- Josh Achiam et al. “Gpt-4 technical report”. In: arXiv preprint arXiv:2303.08774 (2023).
- Jean-Baptiste Alayrac et al. “Flamingo: a Visual Language Model for Few-Shot Learning”. In: Advances in Neural Information Processing Systems 35 (2022), pp. 23716–23736.
- Suzi A Aleksander et al. “The gene ontology knowledgebase in 2023”. In: Genetics 224.1 (2023), iyad031. find
- Joanna S Amberger et al. “OMIM. org: Online Mendelian Inheritance in Man (OMIM®), an online catalog of human genes and genetic disorders”. In: Nucleic acids research 43.D1 (2015), pp. D789–D798. find
- Claudine Badue et al. Self-Driving Cars: A Survey. 2019. arXiv: 1901.04407 [cs.RO]. url: https://arxiv.org/abs/1901.04407. — [arXiv:1901.04407](https://arxiv.org/abs/1901.04407) ↗
- DL Bandalos. Measurement theory and applications for the social sciences. Guilford Publications, 2018. find
- SHIV BANSAL. “The Impact of Artificial Intelligence and Machine Learning on Financial Markets”. In: Innovative Research Thoughts 7.4 (2021), pp. 217–222. find
- Francesco Barbieri et al. “TweetEval:Unified Benchmark and Comparative Evaluation for Tweet Classification”. In: Proceedings of Findings of EMNLP. 2020. — TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
- Emily M. Bender et al. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021, pp. 610–623. find
- Yoshua Bengio. “From systematic generalization to machine intelligence: Benefits and challenges”. In: arXiv preprint arXiv:1911.01547 (2019). find
- Maciej Besta et al. “Graph of Thoughts: Solving Elaborate Problems with Large Language Models”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. 2024, pp. 17682–17690. find
- Kevin Black et al. “Training diffusion models with reinforcement learning”. In: arXiv preprint arXiv:2305.13301 (2023). — [arXiv:2305.13301](https://arxiv.org/abs/2305.13301) find
- Tom Brown et al. “Language models are few-shot learners”. In: Advances in neural information processing systems 33 (2020), pp. 1877–1901.
- Melisa Cahnmann-Taylor and Richard Siegesmund. “Arts-based research in education”. In: Foundations for practice. New York: Routledge (2008). find
- Hongmin Cai et al. “Exploring Multimodal Approaches for Alzheimer’s Disease Detection Using Patient Speech Transcript and Audio Data”. In: arXiv preprint arXiv:2307.02514 (2023). — [arXiv:2307.02514](https://arxiv.org/abs/2307.02514) find
- Wenhu Chen et al. “MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training”. In: arXiv preprint arXiv:2403.09611 (2024). — [arXiv:2403.09611](https://arxiv.org/abs/2403.09611) find
- Paul F Christiano et al. “Deep reinforcement learning from human preferences”. In: Advances in neural information processing systems 30 (2017). find
- Zheng Chu et al. “Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future”. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Ed. by Lun-Wei Ku, Andre Martins, and Vivek Srikumar. B… find
- Haixing Dai et al. “AD-AutoGPT: An Autonomous GPT for Alzheimer’s Disease Infodemiology”. In: arXiv preprint arXiv:2306.10095 (2023). — [arXiv:2306.10095](https://arxiv.org/abs/2306.10095) find
- Haixing Dai et al. “Chataug: Leveraging chatgpt for text data augmentation”. In: arXiv preprint arXiv:2302.13007 (2023).
- Paulo Ney De Souza and Jorge-Nuno Silva. Berkeley Problems in Mathematics. Springer, 1998. find
- Mostafa Dehghani et al. “Scaling vision transformers to 22 billion parameters”. In: International Conference on Machine Learning. PMLR. 2023, pp. 7480–7512. find
- Lizz Denneau. Unveiling the Power of Persona. en. http://web.archive . org / web / 20080207010024/http://www.808multimedia.com/winnt/kernel.htm. ↗
- John Derby. “Disability studies and art education”. In: Studies in Art Education 52.2 (2011), pp. 94–111. find
- Jacob Devlin et al. “Bert: Pre-training of deep bidirectional transformers for language understanding”. In: arXiv preprint arXiv:1810.04805 (2018).
- Alexey Dosovitskiy et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”. In: International Conference on Learning Representations. 2020.
- Fei Dou et al. “Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges”. In: arXiv preprint arXiv:2309.07438 (2023). — [arXiv:2309.07438](https://arxiv.org/abs/2309.07438) find
- Saadia Gutta Essa, Turgay Celik, and Nadia Emelia Human-Hendricks. “Personalized adaptive learning technologies based on machine learning techniques to identify learning styles: A systematic literature review”. In: IEEE Access 11 (2023), pp. 48392–48409. find
- Ying Fan et al. “Reinforcement learning for fine-tuning text-to-image diffusion models”. In: Advances in Neural Information Processing Systems 36 (2024). find
- Steven Y. Feng et al. A Survey of Data Augmentation Approaches for NLP. 2021. arXiv: 2105.03075 [cs.CL]. url: https://arxiv.org/abs/2105.03075. ↗
- Enrique Fernandez et al. Learning ROS for robotics programming. Packt Publishing Ltd, 2015. find
- Simon Frieder et al. “Mathematical capabilities of chatgpt”. In: Advances in neural information processing systems 36 (2024).
- Huan Fu et al. “3d-front: 3d furnished rooms with layouts and semantics”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, pp. 10933–10942. find
- Luyu Gao et al. “Pal: Program-aided language models”. In: International Conference on Machine Learning. PMLR. 2023, pp. 10764–10799. find
- Tianyu Gao, Adam Fisch, and Danqi Chen. “Making pre-trained language models better few-shot learners”. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. 2021, pp. 3816–3830.
- Xinyu Gong et al. “Advancing microbial production through artificial intelligence-aided biology”. In: Biotechnology Advances (2024), p. 108399. find
- Zihan Guan et al. “Cohortgpt: An enhanced gpt for participant recruitment in clinical study”. In: arXiv preprint arXiv:2307.11346 (2023). — [arXiv:2307.11346](https://arxiv.org/abs/2307.11346) find
- Ying Guo and Daniel Lee. “Leveraging chatgpt for enhancing critical thinking skills”. In: Journal of Chemical Education 100.12 (2023), pp. 4876–4883. find
- Kelvin Guu et al. REALM: Retrieval-Augmented Language Model Pre-Training. 2020. arXiv: 2002.08909 [cs.CL]. url: https://arxiv.org/abs/2002.08909. ↗
- Zeyu Han et al. “Parameter-efficient fine-tuning for large models: A comprehensive survey”. In: arXiv preprint arXiv:2403.14608 (2024).
- Kristiina Häyrinen, Kaija Saranto, and Pirkko Nykänen. “Definition, structure, content, use and impacts of electronic health records: a review of the research literature”. In: International journal of medical informatics 77.5 (2008), pp. 291–304. find
- Kaiming He et al. “Masked autoencoders are scalable vision learners”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 16000–16009.
- US Department of Health, Office of the Assistant Secretary for Planning Human Services, and Evaluation. Compilation of State Data on the Affordable Care Act. Accessed: September 19, 2024. 2024. url: https://aspe.hhs.gov/compilation-state-data-affordable-care-act. ↗
- Dan Hendrycks et al. “Measuring massive multitask language understanding”. In: arXiv preprint arXiv:2009.03300 (2020).
- Dan Hendrycks et al. “Measuring Mathematical Problem Solving With the MATH Dataset”. In: NeurIPS (2021).
- William Hersh and David Hickam. “Use of a multi-application computer workstation in a clinical setting.” In: Bulletin of the Medical Library Association 82.4 (1994), p. 382. find
- William Hersh et al. “OHSUMED: An interactive retrieval evaluation and new large test collection for research”. In: SIGIR’94: Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, organised by Dublin City University. Springer.… find
- Jason Holmes et al. “Benchmarking a Foundation Large Language Model on its Ability to Relabel Structure Names in Accordance With the American Association of Physicists in Medicine Task Group-263 Report”. In: Practical Radiation Oncology (2024). find
- Jason Holmes et al. “Benchmarking a foundation LLM on its ability to re-label structure names in accordance with the AAPM TG-263 report”. In: arXiv preprint arXiv:2310.03874 (2023). — [arXiv:2310.03874](https://arxiv.org/abs/2310.03874) find
- Jason Holmes et al. “Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics”. In: arXiv preprint arXiv:2304.01938 (2023). find
- Wenpin Hou and Zhicheng Ji. “GeneTuring tests GPT models in genomics”. In: BioRxiv (2023). find
- Edward J Hu et al. “Lora: Low-rank adaptation of large language models”. In: arXiv preprint arXiv:2106.09685 (2021).
- Xiaoyang Hu et al. “In-context analogical reasoning with pre-trained language models”. In: arXiv preprint arXiv:2305.17626 (2023). find
- Shima Imani, Liang Du, and Harsh Shrivastava. MathPrompter: Mathematical Reasoning using Large Language Models. 2023. arXiv: 2303.05398 [cs.CL]. url: https://arxiv.org/ abs/2303.05398. ↗
- AQ Jiang et al. “Mistral 7B (2023)”. In: arXiv preprint arXiv:2310.06825 (2023). find
- Qiao Jin et al. “Genegpt: Augmenting large language models with domain tools for improved access to biomedical information”. In: Bioinformatics 40.2 (2024), btae075.
- Alistair EW Johnson et al. “MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports”. In: Scientific data 6.1 (2019), p. 317. find
- Alistair EW Johnson et al. “MIMIC-III, a freely accessible critical care database”. In: Scientific data 3.1 (2016), pp. 1–9.
- Sarah Johnson et al. “Ethical Considerations in Multimodal Large Language Models: A Framework for Bias Evaluation and Mitigation”. In: AI Ethics Journal 5.1 (2024), pp. 1–15. find
- Takeshi Kojima et al. “Large language models are zero-shot reasoners”. In: Advances in neural information processing systems 35 (2022), pp. 22199–22213.
- Anis Koubaa et al. “Exploring ChatGPT Capabilities and Limitations: A Survey”. In: IEEE Access 11 (2023), pp. 118698–118721. doi: 10.1109/ACCESS.2023.3326474. — [doi:10.1109/access.2023.3326474](https://doi.org/10.1109/access.2023.3326474) find
- Clemens Scott Kruse et al. “The use of electronic health records to support population health: a systematic review of the literature”. In: Journal of medical systems 42.11 (2018), p. 214. find
- Tom Kubiszyn and Gary D Borich. Educational testing and measurement. John Wiley & Sons, 2024. find
- Brenden M Lake et al. “Building machines that learn and think like people”. In: Behavioral and Brain Sciences 40 (2017). find
- Nathan Lambert. Reverse engineering OpenAI’s o1 — interconnects.ai. https://www.interconnects.ai/p/reverse-engineering-openai-o1. [Accessed 23-09-2024]. ↗
- Serge Lang. Real and Functional Analysis. Springer, 1993. find
- Ehsan Latif, Xiaoming Zhai, and Lei Liu. “Ai gender bias, disparities, and fairness: Does training data matter?” In: arXiv preprint arXiv:2312.10833 (2023). — [arXiv:2312.10833](https://arxiv.org/abs/2312.10833) find
- Hung Le et al. “CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules”. In: The Twelfth International Conference on Learning Representations. 2024. url: https://openreview.net/forum?id=vYhglxSj8j. ↗
- Gyeong-Geon Lee et al. “Multimodality of ai for education: Towards artificial general intelligence”. In: arXiv preprint arXiv:2312.06037 (2023). — [arXiv:2312.06037](https://arxiv.org/abs/2312.06037) find
- Kimin Lee et al. “Aligning text-to-image models using human feedback”. In: arXiv preprint arXiv:2302.12192 (2023). — [arXiv:2302.12192](https://arxiv.org/abs/2302.12192) find
- Brian Lester, Rami Al-Rfou, and Noah Constant. “The Power of Scale for Parameter-Efficient Prompt Tuning”. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021, pp. 3045–3059.
- Brian C Leventhal and Kathryn N Thompson. “Surveying the measurement profession to assist recruiting in the United States”. In: Educational Measurement: Issues and Practice 40.3 (2021), pp. 83–95. find
- Martha Lewis and Melanie Mitchell. “Using counterfactual tasks to evaluate the generality of analogical reasoning in large language models”. In: arXiv preprint arXiv:2402.08955 (2024). — [arXiv:2402.08955](https://arxiv.org/abs/2402.08955) find
- Patrick Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. 2021. arXiv: 2005.11401 [cs.CL]. url: https://arxiv.org/abs/2005.11401. ↗
- Lgw. GitHub - lgw863/LogiQA-dataset. en. url: https://github.com/lgw863/LogiQA-dataset. ↗
- Feng Li et al. “Deep learning-based automated detection of retinal diseases using optical coherence tomography images”. In: Biomedical optics express 10.12 (2019), pp. 6204–6226. find
- Lingyao Li et al. “A scoping review of using Large Language Models (LLMs) to investigate Electronic Health Records (EHRs)”. In: arXiv preprint arXiv:2405.03066 (2024). — [arXiv:2405.03066](https://arxiv.org/abs/2405.03066) find
- Pin-Hui Li et al. “Solving the Self-regulated Learning Problem: Exploring the Performance of ChatGPT in Mathematics”. In: Innovative Technologies and Learning. Ed. by Yueh-Min Huang and Tânia Rocha. Cham: Springer Nature Switzerland, 2023, pp. 77–86. isbn: 978-3-031-40113-8. find
- Wenhai Li et al. “CogVLM: Visual Expert for Pretrained Language Models”. In: arXiv preprint arXiv:2311.03079 (2023).
- Xian Li and Percy Liang. “Prefix-Tuning: Optimizing Continuous Prompts for Generation”. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. 2021, pp. 4582–4597. find
- Xiang Li et al. “Artificial General Intelligence for Medical Imaging”. In: arXiv preprint arXiv:2306.05480 (2023). — [arXiv:2306.05480](https://arxiv.org/abs/2306.05480) find
- Xiaoyuan Li et al. Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction. 2024. arXiv: 2406 . 00755 [cs.CL]. url: https : //arxiv.org/abs/2406.00755. find
- Xiujun Li et al. “Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks”. In: European Conference on Computer Vision. Springer. 2020, pp. 121–137.
- Yiwei Li et al. “ALDM-Grasping: Diffusion-aided Zero-Shot Sim-to-Real Transfer for Robot Grasping”. In: arXiv preprint arXiv:2403.11459 (2024). — [arXiv:2403.11459](https://arxiv.org/abs/2403.11459) find
- Zhening Li et al. LEMMA: Bootstrapping High-Level Mathematical Reasoning with Learned Symbolic Abstractions. 2022. arXiv: 2211.08671 [cs.AI]. url: https://arxiv.org/abs/ 2211.08671. — [arXiv:2211.08671](https://arxiv.org/abs/2211.08671) ↗
- Percy Liang et al. “Holistic Evaluation of Language Models”. In: arXiv preprint arXiv:2211.09110 (2022). find
- Wenxiong Liao et al. “Differentiating ChatGPT-generated and human-written medical texts: quantitative study”. In: JMIR Medical Education 9.1 (2023), e48904. find
- Wenxiong Liao et al. “Mask-guided BERT for few-shot text classification”. In: Neurocomputing (2024), p. 128576. find
- Chengyuan Liu et al. “More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs”. In: arXiv preprint arXiv:2405.17830 (2024). find
- Mingjie Liu et al. “Chipnemo: Domain-adapted llms for chip design”. In: arXiv preprint arXiv:2311.00176 (2023). — [arXiv:2311.00176](https://arxiv.org/abs/2311.00176) find
- Yiheng Liu et al. “Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models”. In: arXiv preprint arXiv:2304.01852 (2023).
- Yiheng Liu et al. Understanding LLMs: A Comprehensive Overview from Training to Inference. 2024. arXiv: 2401.02038 [cs.CL]. url: https://arxiv.org/abs/2401.02038. — [arXiv:2401.02038](https://arxiv.org/abs/2401.02038) ↗
- Yiheng Liu et al. “Understanding llms: A comprehensive overview from training to inference”. In: arXiv preprint arXiv:2401.02038 (2024). — [arXiv:2401.02038](https://arxiv.org/abs/2401.02038) find
- Yinhan Liu et al. “Roberta: A robustly optimized bert pretraining approach”. In: arXiv preprint arXiv:1907.11692 (2019).
- Ze Liu et al. “Swin transformer: Hierarchical vision transformer using shifted windows”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, pp. 10012– 10022.
- Zhengliang Liu et al. “Context Matters: A Strategy to Pre-train Language Model for Science Education”. In: arXiv preprint arXiv:2301.12031 (2023). — [arXiv:2301.12031](https://arxiv.org/abs/2301.12031) find
- Zhengliang Liu et al. “DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4”. In: arXiv preprint arXiv:2303.11032 (2023). — [arXiv:2303.11032](https://arxiv.org/abs/2303.11032) find
- Zhengliang Liu et al. “Evaluating large language models for radiology natural language processing”. In: arXiv preprint arXiv:2307.13693 (2023). — [arXiv:2307.13693](https://arxiv.org/abs/2307.13693) find
- Zhengliang Liu et al. “FINE-TUNING LARGE LANGUAGE MODELS FOR RADIATION ONCOLOGY, A HIGHLY SPECIALIZED HEALTHCARE DOMAIN”. In: International Journal of Particle Therapy 12 (2024), p. 100428. find
- Zhengliang Liu et al. Holistic Evaluation of GPT-4V for Biomedical Imaging. 2023. arXiv: 2312.05256 [eess.IV]. url: https://arxiv.org/abs/2312.05256. — [arXiv:2312.05256](https://arxiv.org/abs/2312.05256) ↗
- Zhengliang Liu et al. “Holistic evaluation of gpt-4v for biomedical imaging”. In: arXiv preprint arXiv:2312.05256 (2023). — [arXiv:2312.05256](https://arxiv.org/abs/2312.05256) find
- Zhengliang Liu et al. “LLM-POTUS Score: A Framework of Analyzing Presidential Debates with Large Language Models”. In: arXiv preprint arXiv:2409.08147 (2024). — [arXiv:2409.08147](https://arxiv.org/abs/2409.08147) find
- Zhengliang Liu et al. “Pharmacygpt: The ai pharmacist”. In: arXiv preprint arXiv:2307.10432 (2023). — [arXiv:2307.10432](https://arxiv.org/abs/2307.10432) find
- Zhengliang Liu et al. “Radiology-llama2: Best-in-class large language model for radiology”. In: arXiv preprint arXiv:2309.06419 (2023). — [arXiv:2309.06419](https://arxiv.org/abs/2309.06419) find
- Zhengliang Liu et al. “RadOnc-GPT: A Large Language Model for Radiation Oncology”. In: arXiv preprint arXiv:2309.10160 (2023). — [arXiv:2309.10160](https://arxiv.org/abs/2309.10160) find
- Zhengliang Liu et al. “Surviving ChatGPT in healthcare”. In: Frontiers in Radiology 3 (2024), p. 1224682. find
- Zhengliang Liu et al. “The Radiation Oncology NLP Database”. In: arXiv preprint arXiv:2401.10995 (2024). — [arXiv:2401.10995](https://arxiv.org/abs/2401.10995) find
- Zhengliang Liu et al. “Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities”. In: arXiv preprint arXiv:2310.19626 (2023). — [arXiv:2310.19626](https://arxiv.org/abs/2310.19626) find
- Penfold Louisa. Make a cardboard construction. 2020. url: https://louisapenfold.com/ cardboard-art/. ↗
- Jiasen Lu et al. “ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks”. In: Advances in Neural Information Processing Systems. 2019.
- Yanjun Lyu et al. “GP-GPT: Large Language Model for Gene-Phenotype Mapping”. In: arXiv preprint arXiv:2409.09825 (2024). — [arXiv:2409.09825](https://arxiv.org/abs/2409.09825) find
- Chong Ma et al. “An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT”. In: IEEE Transactions on Artificial Intelligence (2024). find
- Jianzhu Ma et al. “Using deep learning to model the hierarchical structure and function of a cell”. In: Nature methods 15.4 (2018), pp. 290–298. find
- Andrew Maas et al. “Learning word vectors for sentiment analysis”. In: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies. 2011, pp. 142–150.
- Aman Madaan et al. “Self-Refine: Iterative Refinement with Self-Feedback”. In: Thirty-seventh Conference on Neural Information Processing Systems. 2023. url: https://openreview.net/forum?id=S37hOerQLB. ↗
- Timothy R McIntosh et al. “Inadequacies of large language model benchmarks in the era of generative artificial intelligence”. In: arXiv preprint arXiv:2402.09880 (2024). — [arXiv:2402.09880](https://arxiv.org/abs/2402.09880) find
- Ninareh Mehrabi et al. A Survey on Bias and Fairness in Machine Learning. 2022. arXiv: 1908.09635 [cs.LG]. url: https://arxiv.org/abs/1908.09635. ↗
- Metadocs. How OpenAI o1 works in a simple way and why it matters for RAG and Agentic -Use AI the right way — metadocs.co. https://www.metadocs.co/2024/09/17/how-openai-o1-works-in-a-simple-way-and-why-it-matters-for-rag-and-agentic/. [Accessed 23-09-2024]. ↗
- Shubhra Mishra et al. MathCAMPS: Fine-grained Synthesis of Mathematical Problems From Human Curricula. 2024. arXiv: 2407.00900 [cs.AI]. url: https://arxiv.org/abs/2407. 00900. ↗
- Melanie Mitchell. “Abstraction and analogy-making in artificial intelligence”. In: Annals of the New York Academy of Sciences 1505.1 (2021), pp. 79–101.
- Moran Mizrahi et al. “State of what art? a call for multi-prompt llm evaluation”. In: Transactions of the Association for Computational Linguistics 12 (2024), pp. 933–949. find
- Saif Mohammad et al. “SemEval-2018 Task 1: Affect in Tweets”. In: Proceedings of the 12th International Workshop on Semantic Evaluation. New Orleans, Louisiana: Association for Computational Linguistics, June 2018, pp. 1–17. doi: 10.18653/v1/S18- 1001. url: https://aclanthology.org/S18-1001. ↗
- Sam Musker et al. Semantic Structure-Mapping in LLM and Human Analogical Reasoning. 2024. arXiv: 2406.13803 [cs.CL]. url: https://arxiv.org/abs/2406.13803. ↗
- Shashi Narayan, Shay B. Cohen, and Mirella Lapata. “Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization”. In: ArXiv abs/1808.08745 (2018).
- Yixin Nie et al. “Adversarial NLI: A new benchmark for natural language understanding”. In: arXiv preprint arXiv:1910.14599 (2019).
- Tim Nuttle et al. Cherokee Corpus. https://cherokeedictionary.net/corpus/corpusMain. Accessed: 2024-09-17. 2024. ↗
- OpenAI. “GPT-4 Technical Report”. In: arXiv preprint arXiv:2303.08774 (2023).
- OpenAI. Introducing ChatGPT — openai.com. https://openai.com/blog/chatgpt. [Accessed 28-08-2023]. ↗
- OpenAI. Learning to Reason with LLMs. https://openai.com/index/learning - to -reason-with-llms/. [Accessed 19-09-2024]. 2024. ↗
- Long Ouyang et al. “Training language models to follow instructions with human feedback”. In: Advances in neural information processing systems 35 (2022), pp. 27730–27744.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. “Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering”. In: Conference on health, inference, and learning. PMLR. 2022, pp. 248–260. — MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
- Ronald Carl Petersen et al. “Alzheimer’s disease neuroimaging initiative (ADNI): clinical characterization”. In: Neurology 74 (2010). find
- Pikachu345. [Math Talk] Is the latest AI model’s capability approaching that of a PhD in Mathematics? https://www.bilibili.com/video/BV1T644e1EMN/. [Online; accessed 19-Sep-2024]. Bilibili, 2024. ↗
- William F Pinar. “Currere”. In: Key concepts in curriculum studies. Routledge, 2019, pp. 50– 52. find
- Silviu Pitis et al. “Boosted Prompt Ensembles for Large Language Models”. In: arXiv e-prints (2023), arXiv–2304. find
- Vagelis Plevris, George Papazafeiropoulos, and Alejandro Jiménez Rios. “Chatbots Put to the Test in Math and Logic Problems: A Comparison and Assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard”. In: AI 4.4 (2023), pp. 949–969. issn: 2673-2688. doi: 10.3390/ai4040048. url: https://www.mdpi.com/267… — [doi:10.3390/ai4040048](https://doi.org/10.3390/ai4040048) ↗
- Gabriel Poesia and Noah D. Goodman. “Peano: learning formal mathematical reasoning”. In: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 381.2251 (2023), p. 20220044. doi: 10 . 1098 / rsta . 2022 . 0044. eprint: https://royalsocietypublishing .… find
- Gabriel Poesia et al. Learning Formal Mathematics From Intrinsic Motivation. 2024. arXiv: 2407.00695 [cs.AI]. url: https://arxiv.org/abs/2407.00695. ↗
- Maria Pontiki et al. “SemEval-2014 Task 4: Aspect Based Sentiment Analysis”. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics, Aug. 2014, pp. 27–35.
- Maria Pontiki et al. “SemEval-2015 Task 12: Aspect Based Sentiment Analysis”. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). Association for Computational Linguistics, June 2015, pp. 486–495.
- Indrasen Poola. Tuning ChatGPT Mathematical Reasoning Limitations and Failures with Process Supervision. Zenodo. June 2023. doi: 10 . 5281 / zenodo . 8296440. url: https : //doi.org/10.5281/zenodo.8296440. — [doi:10.5281/zenodo.8296440](https://doi.org/10.5281/zenodo.8296440) find
- Guanghui Qin and Jason Eisner. “Learning How to Ask: Querying LMs with Mixtures of Soft Prompts”. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics. 2021, pp. 5203–5212.
- Alec Radford et al. “Improving language understanding by generative pre-training”. In: OpenAI (2018). find
- Alec Radford et al. “Language models are unsupervised multitask learners”. In: OpenAI blog 1.8 (2019), p. 9. find
- Alec Radford et al. “Learning transferable visual models from natural language supervision”. In: International conference on machine learning. PMLR. 2021, pp. 8748–8763.
- Jack W Rae et al. “Scaling language models: Methods, analysis & insights from training gopher”. In: arXiv preprint arXiv:2112.11446 (2021). find
- Rafael Rafailov et al. “Direct preference optimization: Your language model is secretly a reward model”. In: Advances in Neural Information Processing Systems 36 (2024).
- Steve Rathje et al. “GPT is an effective tool for multilingual psychological text analysis”. In: Proceedings of the National Academy of Sciences 121.34 (2024), e2308950121. find
- Saed Rezayi et al. “AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition”. In: International Joint Conference on Artificial Intelligence. 2022. find
- Saed Rezayi et al. “ClinicalRadioBERT: Knowledge-Infused Few Shot Learning for Clinical Notes Named Entity Recognition”. In: Machine Learning in Medical Imaging: 13th International Workshop, MLMI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings. Springer. 2022,… find
- Saed Rezayi et al. “Exploring new frontiers in agricultural nlp: Investigating the potential of large language models for food applications”. In: IEEE Transactions on Big Data (2024). find
- Alexey Romanov and Chaitanya Shivade. “Lessons from natural language inference in the clinical domain”. In: arXiv preprint arXiv:1808.06752 (2018).
- Sara Rosenthal, Noura Farra, and Preslav Nakov. “SemEval-2017 Task 4: Sentiment Analysis in Twitter”. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Vancouver, Canada: Association for Computational Linguistics, Aug. 2017, pp. 502–518. doi: 10.18653/v1/S17-2…
- Sheldon M Ross. Stochastic processes. John Wiley & Sons, 1995. find
- Zhao Ruixue et al. “Insights and reflections of the impact of ChatGPT on intelligent knowledge services in libraries”. In: Journal of Library and Information Sciences in Agriculture 35.1 (2023), p. 29. find
- Luis M. Sánchez-Ruiz et al. “ChatGPT Challenges Blended Learning Methodologies in Engineering Education: A Case Study in Mathematics”. In: Applied Sciences 13.10 (2023). issn: 2076-3417. doi: 10.3390/app13106039. url: https://www.mdpi.com/2076-3417/13/ 10/6039. — [doi:10.3390/app13106039](https://doi.org/10.3390/app13106039) ↗
- Gargi Sathe et al. “Cross-Language Assessment of Mathematical Capability of ChatGPT”. In: Intelligent Systems Design and Applications. Ed. by Ajith Abraham et al. Cham: Springer Nature Switzerland, 2024, pp. 468–479. isbn: 978-3-031-64779-6. find
- John Schulman et al. “Proximal policy optimization algorithms”. In: arXiv preprint arXiv:1707.06347 (2017). find
- Emily Sheng et al. “Revealing the Biases in Large Language Models”. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021, pp. 932–947. find
- Olena Shevtsova et al. “The role of art education in personality development”. In: Amazonia Investiga 13.74 (2024), pp. 143–155. find
- Yucheng Shi et al. “MedEdit: Model Editing for Medical Question Answering with External Knowledge Bases”. In: arXiv preprint arXiv:2309.16035 (2023). — [arXiv:2309.16035](https://arxiv.org/abs/2309.16035) find
- Peng Shu et al. “LLMs for Coding and Robotics Education”. In: arXiv preprint arXiv:2402.06116 (2024). — [arXiv:2402.06116](https://arxiv.org/abs/2402.06116) find
- Kashun Shum, Shizhe Diao, and Tong Zhang. “Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data”. In: Findings of the Association for Computational Linguistics: EMNLP 2023. Ed. by Houda Bouamor, Juan Pino, and Kalika Bali. Singapore: Association for Computational Lingu…
- Shelley Staples and Bradley Dilger. Corpus and repository of writing. https://crow.corporaproject.org. Learner corpus articulated with repository. 2018. ↗
- Nisan Stiennon et al. “Learning to summarize with human feedback”. In: Advances in Neural Information Processing Systems 33 (2020), pp. 3008–3021. find
- Amber Stubbs and Özlem Uzuner. “Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus”. In: Journal of biomedical informatics 58 (2015), S20–S29. find
- Lichao Sun et al. “Trustllm: Trustworthiness in large language models”. In: arXiv preprint arXiv:2401.05561 (2024).
- Colleen M. Suter et al. “The Affordable Care Act at 10 Years: Evaluating the Evidence and Navigating an Uncertain Future”. In: Journal of Surgical Research 258 (2021), pp. 207–220. url: https://www.journalofsurgicalresearch.com/article/S0022- 4804(21)00016-0/fulltext. ↗
- Chenjiao Tan et al. “On the promises and challenges of multimodal foundation models for geographical, environmental, agricultural, and urban planning applications”. In: arXiv preprint arXiv:2312.17016 (2023). — [arXiv:2312.17016](https://arxiv.org/abs/2312.17016) find
- Songbo Tan. “Large margin DragPushing strategy for centroid text categorization”. In: Expert Systems with Applications 33.1 (2007), pp. 215–220. find
- Chenhao Tang et al. “PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models”. In: arXiv preprint arXiv:2309.10238 (2023). find
- Jie Tian et al. “Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding”. In: arXiv preprint arXiv:2401.12983 (2024). — [arXiv:2401.12983](https://arxiv.org/abs/2401.12983) find
- Hugo Touvron et al. “Llama 2: Open foundation and fine-tuned chat models”. In: arXiv preprint arXiv:2307.09288 (2023).
- Hugo Touvron et al. “Training data-efficient image transformers & distillation through attention”. In: International Conference on Machine Learning. PMLR. 2021, pp. 10347–10357.
- Cynthia Van Hee, Els Lefever, and Véronique Hoste. “Semeval-2018 task 3: Irony detection in english tweets”. In: Proceedings of the 12th international workshop on semantic evaluation. 2018, pp. 39–50.
- Ashish Vaswani et al. “Attention is all you need”. In: Advances in neural information processing systems 30 (2017).
- Oriol Vinyals et al. “Grandmaster level in StarCraft II using multi-agent reinforcement learning”. In: nature 575.7782 (2019), pp. 350–354. find
- Tu Vu, Harsh Jhamtani, and Eduard Hovy. “SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer”. In: arXiv preprint arXiv:2110.07904 (2021).
- Alex Wang et al. “GLUE: A multi-task benchmark and analysis platform for natural language understanding”. In: arXiv preprint arXiv:1804.07461 (2018).
- Hui Wang and Anh Dang. “Enhancing L2 Writing with Generative AI: A Systematic Review of Pedagogical Integration and Outcomes”. In: preprint DOI: 10.13140/RG.2.2.19572.16005 (2024). — [doi:10.13140/rg.2.2.19572.16005](https://doi.org/10.13140/rg.2.2.19572.16005) find
- Hui Wang et al. “Seeing ChatGPT Through Universities’ Policies, Resources and Guidelines”. In: arXiv preprint arXiv:2312.05235 (2023). find
- Jiaqi Wang et al. “A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks”. In: arXiv preprint arXiv:2408.01319 (2024). — [arXiv:2408.01319](https://arxiv.org/abs/2408.01319) find
- Jiaqi Wang et al. “Large language models for robotics: Opportunities, challenges, and perspectives”. In: arXiv preprint arXiv:2401.04334 (2024). — [arXiv:2401.04334](https://arxiv.org/abs/2401.04334) find
- Jiaqi Wang et al. “Prompt engineering for healthcare: Methodologies and applications”. In: arXiv preprint arXiv:2304.14670 (2023). — [arXiv:2304.14670](https://arxiv.org/abs/2304.14670) find
- Lei Wang et al. “A survey on large language model based autonomous agents”. In: Frontiers of Computer Science 18.6 (2024), p. 186345.
- Tingwu Wang et al. “Benchmarking model-based reinforcement learning”. In: arXiv preprint arXiv:1907.02057 (2019). find
- Xuezhi Wang et al. “Self-consistency improves chain of thought reasoning in language models”. In: arXiv preprint arXiv:2203.11171 (2022).
- Yizhong Wang et al. “Efficient Multimodal Large Language Models with Pruning-Based Compression”. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023, pp. 10112–10125. find
- Yousef Wardat et al. “ChatGPT: A revolutionary tool for teaching and learning mathematics”. In: Eurasia Journal of Mathematics, Science and Technology Education 19.7 (2023). Publisher: Modestum Ltd. issn: 1305-8215, 1305-8223. doi: 10.29333/ejmste/13272. url: https://www.ejmste . com / article… — [doi:10.29333/ejmste/13272](https://doi.org/10.29333/ejmste/13272) ↗
- Taylor Webb, Keith J Holyoak, and Hongjing Lu. “Emergent analogical reasoning in large language models”. In: Nature Human Behaviour 7.9 (2023), pp. 1526–1541. find
- Jason Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. 2023. arXiv: 2201.11903 [cs.CL]. url: https://arxiv.org/abs/2201.11903. ↗
- Jason Wei et al. “Chain-of-thought prompting elicits reasoning in large language models”. In: Advances in Neural Information Processing Systems (NeurIPS). 2022.
- Yaonai Wei et al. “Chat2Brain: A Method for Mapping Open-Ended Semantic Queries to Brain Activation Maps”. In: arXiv preprint arXiv:2309.05021 (2023). — [arXiv:2309.05021](https://arxiv.org/abs/2309.05021) find
- Johannes Welbl, Nelson F. Liu, and Matt Gardner. “Crowdsourcing multiple choice science questions”. In: arXiv preprint arXiv:1707.06209 (2017). url: https://arxiv.org/abs/1707. 06209. ↗
- Adina Williams, Nikita Nangia, and Samuel R Bowman. “A broad-coverage challenge corpus for sentence understanding through inference”. In: arXiv preprint arXiv:1704.05426 (2017).
- Zihao Wu et al. “Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task”. In: arXiv preprint arXiv:2304.09138 (2023). find
- Shaochen Xu et al. “Reasoning before comparison: LLM-enhanced semantic similarity metrics for domain specialized text analysis”. In: arXiv preprint arXiv:2402.11398 (2024). — [arXiv:2402.11398](https://arxiv.org/abs/2402.11398) find
- Yixuan Yang et al. LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model. 2024. arXiv: 2406.03866 [cs.CV]. url: https://arxiv.org/abs/2406. 03866. — [arXiv:2406.03866](https://arxiv.org/abs/2406.03866) ↗
- Zhenyu Yang et al. “Multimodal Reasoning Benchmark: Integrating Vision and Language for Complex Problem Solving”. In: arXiv preprint arXiv:2402.12425 (2024). — [arXiv:2402.12425](https://arxiv.org/abs/2402.12425) find
- Shunyu Yao et al. “ReAct: Synergizing reasoning and acting in language models”. In: arXiv preprint arXiv:2210.03629 (2022).
- Shunyu Yao et al. “Tree of thoughts: Deliberate problem solving with large language models”. In: arXiv preprint arXiv:2305.10601 (2023). find
- Michael Ku Yu et al. “Translation of genotype to phenotype by a hierarchy of cell subsystems”. In: Cell systems 2.2 (2016), pp. 77–88. find
- Joy He-Yueya et al. Solving Math Word Problems by Combining Language Models With Symbolic Solvers. 2023. arXiv: 2304.09102 [cs.CL]. url: https://arxiv.org/abs/2304. 09102. ↗
- yymYM. Stock Trading QA: A Question Answering Dataset for Finance and Trading. https://huggingface.co/datasets/yymYM/stock_trading_QA. Accessed: 2024-09-17. 2023. ↗
- Marcos Zampieri et al. “SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)”. In: Proceedings of the 13th International Workshop on Semantic Evaluation. Ed. by Jonathan May et al. Minneapolis, Minnesota, USA: Association for Computational Linguistics, Ju… — [ACL S19-2010](https://aclanthology.org/S19-2010/)
- Eric Zelikman et al. “Quiet-star: Language models can teach themselves to think before speaking”. In: arXiv preprint arXiv:2403.09629 (2024).
- Kai Zhang et al. “A generalist vision–language foundation model for diverse biomedical tasks”. In: Nature Medicine (2024), pp. 1–13. find
- Kai Zhang et al. “BiomedGPT: a unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks”. In: arXiv preprint arXiv:2305.17100 (2023). find
- Qiang Zhang et al. “mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality”. In: arXiv preprint arXiv:2304.14178 (2023).
- Shengyu Zhang et al. “Instruction tuning for large language models: A survey”. In: arXiv preprint arXiv:2308.10792 (2023).
- Tiezheng Zhang et al. “Leveraging AI Predicted and Expert Revised Annotations in Interactive Segmentation: Continual Tuning or Full Training?” In: arXiv preprint arXiv:2402.19423 (2024). — [arXiv:2402.19423](https://arxiv.org/abs/2402.19423) find
- Yutong Zhang et al. Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports. 2024. arXiv: 2407.05758 [eess.IV]. url: https://arxiv.org/abs/2407.05758. — [arXiv:2407.05758](https://arxiv.org/abs/2407.05758) ↗
- Zhuosheng Zhang et al. “Automatic Chain of Thought Prompting in Large Language Models”. In: The Eleventh International Conference on Learning Representations. 2023. url: https://openreview.net/forum?id=5NTt8GFjUHkr. ↗
- Huan Zhao et al. “Ophtha-llama2: A large language model for ophthalmology”. In: arXiv preprint arXiv:2312.04906 (2023). — [arXiv:2312.04906](https://arxiv.org/abs/2312.04906) find
- Huaqin Zhao et al. “Revolutionizing finance with llms: An overview of applications and insights”. In: arXiv preprint arXiv:2401.11641 (2024).
- Lin Zhao et al. “When brain-inspired ai meets agi”. In: arXiv preprint arXiv:2303.15935 (2023). — [arXiv:2303.15935](https://arxiv.org/abs/2303.15935) find
- Tianyang Zhong et al. ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data. 2023. arXiv: 2310.05242 [cs.CL]. url: https://arxiv.org/abs/2310.05242. — [arXiv:2310.05242](https://arxiv.org/abs/2310.05242) ↗
- Ce Zhou et al. “A comprehensive survey on pretrained foundation models: A history from bert to chatgpt”. In: arXiv preprint arXiv:2302.09419 (2023). find
- Daniel M Ziegler et al. “Fine-tuning language models from human preferences”. In: arXiv preprint arXiv:1909.08593 (2019). find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhong-2024-evaluation-openai-o1-opportunities-challenges/