GPT-4 Technical Report

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

10299 papers of our library cite it (of the 140673 whose bibliographies we read); 383 of them have a page here, the first 100 below, 9916 not yet.

• Fanar: An Arabic-Centric Multimodal Generative AI Platform
• Towards an Integrated Methodology of Dating Biblical Texts: The Case of the Book of Jeremiah
• L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
• Speech Act Patterns for Improving Generalizability of Explainable Politeness Detection Models
• SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
• The Falcon Series of Open Language Models
• Detecting Language Model Attacks with Perplexity
• Cutting Through the Noise: Boosting LLM Performance on Math Word Problems
• PaLM 2 Technical Report
• Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• A General Theoretical Paradigm to Understand Learning from Human Preferences
• Llemma: An Open Language Model For Mathematics
• ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
• Qwen Technical Report
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
• Eliciting Latent Predictions from Transformers with the Tuned Lens
• Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
• InternLM2 Technical Report
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
• PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
• A Survey on Evaluation of Large Language Models
• Jailbreaking Black Box Large Language Models in Twenty Queries
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
• AlpaGasus: Training A Better Alpaca with Fewer Data
• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
• InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
• MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
• Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic
• TheoremQA: A Theorem-driven Question Answering Dataset
• AlphaMath Almost Zero: Process Supervision without Process
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
• FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
• Qwen2-Audio Technical Report
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
• LLM-Assisted Rule Based Machine Translation for Low/No-Resource Languages
• Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture
• Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca
• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
• InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
• Safe RLHF: Safe Reinforcement Learning from Human Feedback
• Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
• DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models
• Mind2Web: Towards a Generalist Agent for the Web
• Multilingual Jailbreak Challenges in Large Language Models
• Investigating Data Contamination in Modern Benchmarks for Large Language Models
• Is GPT-3 a Good Data Annotator?
• Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
• LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
• Generation and Evaluation of English Grammar Multiple-Choice Cloze Exercises
• RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
• A Survey on In-context Learning
• AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
• Faith and Fate: Limits of Transformers on Compositionality
• TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
• LawBench: Benchmarking Legal Knowledge of Large Language Models
• From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
• The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation
• Mathematical Capabilities of ChatGPT
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
• Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
• Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
• IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
• Bias and Fairness in Large Language Models: A Survey
• Enabling Large Language Models to Generate Text with Citations
• Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
• Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
• Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
• Scaling Synthetic Data Creation with 1,000,000,000 Personas
• Gemini: A Family of Highly Capable Multimodal Models
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• Time Travel in LLMs: Tracing Data Contamination in Large Language Models
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
• MiniLLM: On-Policy Distillation of Large Language Models
• CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
• Evaluating Large Language Models: A Comprehensive Survey
• FOLIO: Natural Language Reasoning with First-Order Logic

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.08774 · arXiv 2303.08774

@misc{openai2023gpt,
  title = {{GPT-4 Technical Report}},
  author = {OpenAI and Achiam, Josh and Adler, Steven and Agarwal, Sandhini and Ahmad, Lama and Akkaya, Ilge and Aleman, Florencia Leoni and Almeida, Diogo and Altenschmidt, Janko and Altman, Sam and Anadkat, Shyamal and Avila, Red and Babuschkin, Igor and Balaji, Suchir and Balcom, Valerie and Baltescu, Paul and Bao, Haiming and Bavarian, Mohammad and Belgum, Jeff and Bello, Irwan and Berdine, Jake and Bernadett-Shapiro, Gabriel and Berner, Christopher and Bogdonoff, Lenny and Boiko, Oleg and Boyd, Madelaine and Brakman, Anna-Luisa and Brockman, Greg and Brooks, Tim and Brundage, Miles and Button, Kevin and Cai, Trevor and Campbell, Rosie and Cann, Andrew and Carey, Brittany and Carlson, Chelsea and Carmichael, Rory and Chan, Brooke and Chang, Che and Chantzis, Fotis and Chen, Derek and Chen, Sully and Chen, Ruby and Chen, Jason and Chen, Mark and Chess, Ben and Cho, Chester and Chu, Casey and Chung, Hyung Won and Cummings, Dave and Currier, Jeremiah and Dai, Yunxing and Decareaux, Cory and Degry, Thomas and Deutsch, Noah and Deville, Damien and Dhar, Arka and Dohan, David and Dowling, Steve and Dunning, Sheila and Ecoffet, Adrien and Eleti, Atty and Eloundou, Tyna and Farhi, David and Fedus, Liam and Felix, Niko and Fishman, Simón Posada and Forte, Juston and Fulford, Isabella and Gao, Leo and Georges, Elie and Gibson, Christian and Goel, Vik and Gogineni, Tarun and Goh, Gabriel and Gontijo-Lopes, Rapha and Gordon, Jonathan and Grafstein, Morgan and Gray, Scott and Greene, Ryan and Gross, Joshua and Gu, Shixiang Shane and Guo, Yufei and Hallacy, Chris and Han, Jesse and Harris, Jeff and He, Yuchen and Heaton, Mike and Heidecke, Johannes and Hesse, Chris and Hickey, Alan and Hickey, Wade and Hoeschele, Peter and Houghton, Brandon and Hsu, Kenny and Hu, Shengli and Hu, Xin and Huizinga, Joost and Jain, Shantanu and Jain, Shawn and Jang, Joanne and Jiang, Angela and Jiang, Roger and Jin, Haozhun and Jin, Denny and Jomoto, Shino and Jonn, Billie and Jun, Heewoo and Kaftan, Tomer and Kaiser, Łukasz and Kamali, Ali and Kanitscheider, Ingmar and Keskar, Nitish Shirish and Khan, Tabarak and Kilpatrick, Logan and Kim, Jong Wook and Kim, Christina and Kim, Yongjik and Kirchner, Jan Hendrik and Kiros, Jamie and Knight, Matt and Kokotajlo, Daniel and Kondraciuk, Łukasz and Kondrich, Andrew and Konstantinidis, Aris and Kosic, Kyle and Krueger, Gretchen and Kuo, Vishal and Lampe, Michael and Lan, Ikai and Lee, Teddy and Leike, Jan and Leung, Jade and Levy, Daniel and Li, Chak Ming and Lim, Rachel and Lin, Molly and Lin, Stephanie and Litwin, Mateusz and Lopez, Theresa and Lowe, Ryan and Lue, Patricia and Makanju, Anna and Malfacini, Kim and Manning, Sam and Markov, Todor and Markovski, Yaniv and Martin, Bianca and Mayer, Katie and Mayne, Andrew and McGrew, Bob and McKinney, Scott Mayer and McLeavey, Christine and McMillan, Paul and McNeil, Jake and Medina, David and Mehta, Aalok and Menick, Jacob and Metz, Luke and Mishchenko, Andrey and Mishkin, Pamela and Monaco, Vinnie and Morikawa, Evan and Mossing, Daniel and Mu, Tong and Murati, Mira and Murk, Oleg and Mély, David and Nair, Ashvin and Nakano, Reiichiro and Nayak, Rajeev and Neelakantan, Arvind and Ngo, Richard and Noh, Hyeonwoo and Ouyang, Long and O'Keefe, Cullen and Pachocki, Jakub and Paino, Alex and Palermo, Joe and Pantuliano, Ashley and Parascandolo, Giambattista and Parish, Joel and Parparita, Emy and Passos, Alex and Pavlov, Mikhail and Peng, Andrew and Perelman, Adam and Peres, Filipe de Avila Belbute and Petrov, Michael and Pinto, Henrique Ponde de Oliveira and Michael and Pokorny and Pokrass, Michelle and Pong, Vitchyr H. and Powell, Tolly and Power, Alethea and Power, Boris and Proehl, Elizabeth and Puri, Raul and Radford, Alec and Rae, Jack and Ramesh, Aditya and Raymond, Cameron and Real, Francis and Rimbach, Kendra and Ross, Carl and Rotsted, Bob and Roussez, Henri and Ryder, Nick and Saltarelli, Mario and Sanders, Ted and Santurkar, Shibani and Sastry, Girish and Schmidt, Heather and Schnurr, David and Schulman, John and Selsam, Daniel and Sheppard, Kyla and Sherbakov, Toki and Shieh, Jessica and Shoker, Sarah and Shyam, Pranav and Sidor, Szymon and Sigler, Eric and Simens, Maddie and Sitkin, Jordan and Slama, Katarina and Sohl, Ian and Sokolowsky, Benjamin and Song, Yang and Staudacher, Natalie and Such, Felipe Petroski and Summers, Natalie and Sutskever, Ilya and Tang, Jie and Tezak, Nikolas and Thompson, Madeleine B. and Tillet, Phil and Tootoonchian, Amin and Tseng, Elizabeth and Tuggle, Preston and Turley, Nick and Tworek, Jerry and Uribe, Juan Felipe Cerón and Vallone, Andrea and Vijayvergiya, Arun and Voss, Chelsea and Wainwright, Carroll and Wang, Justin Jay and Wang, Alvin and Wang, Ben and Ward, Jonathan and Wei, Jason and Weinmann, CJ and Welihinda, Akila and Welinder, Peter and Weng, Jiayi and Weng, Lilian and Wiethoff, Matt and Willner, Dave and Winter, Clemens and Wolrich, Samuel and Wong, Hannah and Workman, Lauren and Wu, Sherwin and Wu, Jeff and Wu, Michael and Xiao, Kai and Xu, Tao and Yoo, Sarah and Yu, Kevin and Yuan, Qiming and Zaremba, Wojciech and Zellers, Rowan and Zhang, Chong and Zhang, Marvin and Zhao, Shengjia and Zheng, Tianhao and Zhuang, Juntang and Zhuk, William and Zoph, Barret},
  year = {2023},
  doi = {10.48550/arxiv.2303.08774},
  eprint = {2303.08774},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.08774}
}

References

105 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Tamkin, M. Brundage, J. Clark, and D. Ganguli, “Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models,” Feb. 2021. find
2. “Introducing the new Bing.” https://www.bing.com/new. ↗
3. J. Hilton, R. Nakano, S. Balaji, and J. Schulman, “WebGPT: Improving the factual accuracy of language models through web browsing.” https://openai.com/research/webgpt, Dec. 2021. ↗
4. “ACT-1: Transformer for Actions – Adept.” https://www.adept.ai/blog/act-1. ↗
5. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cum… find our card
6. L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, Z. Kenton, S. Brown, W. Hawkins, T. Stepleton, C. Biles, A. Birhane, J. Haas, L. Rimell, L. A. Hendricks, W. Isaac, S. Legassick, G. Irving, and I. Gabriel, “Ethical and social risks o… find our card
7. I. Solaiman, M. Brundage, J. Clark, A. Askell, A. Herbert-Voss, J. Wu, A. Radford, G. Krueger, J. W. Kim, S. Kreps, M. McCain, A. Newhouse, J. Blazakis, K. McGuffie, and J. Wang, “Release Strategies and the Social Impacts of Language Models,” Nov. 2019. find our card
8. A. Radford, “Improving language understanding with unsupervised learning.” https://ope-nai.com/research/language-unsupervised, June 2018. ↗
9. A. Radford, J. Wu, D. Amodei, D. Amodei, J. Clark, M. Brundage, I. Sutskever, A. Askell, D. Lansky, D. Hernandez, and D. Luan, “Better language models and their implications.” https://openai.com/research/better-language-models, Feb. 2019. ↗
10. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… find
11. S. Altman, “Planning for AGI and beyond.” https://openai.com/blog/planning-for-agi-and-beyond, Feb. 2023. ↗
12. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,”… find
13. P. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Feb. 2023. find
14. M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, “Model Cards for Model Reporting,” in Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 220–229, Jan. 2019. find
15. N. Green, C. Procope, A. Cheema, and A. Adediji, “System Cards, a new resource for understanding how AI systems work.” https://ai.facebook.com/blog/system-cards-a-new-resource-for-understanding-how-ai-systems-work/, Feb. 2022. ↗
16. “DALL·E 2 Preview - Risks and Limitations.” OpenAI, Apr. 2022. find
17. J. Sandbrink, H. Hobbs, J. Swett, A. Dafoe, and A. Sandberg, “Differential Technology Development: A Responsible Innovation Principle for Navigating Technology Risks,” Sept. 2022. find
18. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, N. Joseph, S. Kadavath, J. Kernion, T. Conerly, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, D. Hernandez, T. Hume, S. Johnston, S. Kravec, L. Lovitt, N. Nanda, C. Olsson, D. Amodei, T. Brown, J.… find
19. E. Perez, S. Ringer, K. Lukošiūtė, K. Nguyen, E. Chen, S. Heiner, C. Pettit, C. Olsson, S. Kundu, S. Kadavath, A. Jones, A. Chen, B. Mann, B. Israel, B. Seethor, C. McKinnon, C. Olah, D. Yan, D. Amodei, D. Amodei, D. Drain, D. Li, E. Tran-Johnson, G. Khundadze, J. Kernion, J. Landis, J. Kerr, J. M… find our card
20. B. P. Kehoe, Zen and the Art of the Internet. Project Gutenberg, June 1992. find
21. M. Brundage, K. Mayer, T. Eloundou, S. Agarwal, S. Adler, G. Krueger, J. Leike, and P. Mishkin, “Lessons learned on language model safety and misuse.” https://ope-nai.com/research/language-model-safety-and-misuse, Mar. 2022. ↗
22. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” 2019. find
23. G. C. Bowker and S. L. Star, Sorting Things Out. MIT Press, Aug. 2000. find
24. L. Weidinger, J. Uesato, M. Rauh, C. Griffin, P.-S. Huang, J. Mellor, A. Glaese, M. Cheng, B. Balle, A. Kasirzadeh, C. Biles, S. Brown, Z. Kenton, W. Hawkins, T. Stepleton, A. Birhane, L. A. Hendricks, L. Rimell, W. Isaac, J. Haas, S. Legassick, G. Irving, and I. Gabriel, “Taxonomy of Risks posed by… find
25. I. Solaiman and C. Dennison, “Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets,” Nov. 2021. find
26. H. Khlaaf, “Toward Comprehensive Risk Assessments and Assurance of AI-Based Systems,” Trail of Bits, 2023. find
27. M. Brundage, S. Avin, J. Wang, H. Belfield, G. Krueger, G. Hadfield, H. Khlaaf, J. Yang, H. Toner, R. Fong, T. Maharaj, P. W. Koh, S. Hooker, J. Leung, A. Trask, E. Bluemke, J. Lebensold, C. O’Keefe, M. Koren, T. Ryffel, J. B. Rubinovitz, T. Besiroglu, F. Carugati, J. Clark, P. Eckersley, S. de Haas… find
28. D. Ganguli, L. Lovitt, J. Kernion, A. Askell, Y. Bai, S. Kadavath, B. Mann, E. Perez, N. Schiefer, K. Ndousse, A. Jones, S. Bowman, A. Chen, T. Conerly, N. DasSarma, D. Drain, N. Elhage, S. El-Showk, S. Fort, Z. Hatfield-Dodds, T. Henighan, D. Hernandez, T. Hume, J. Jacobson, S. Johnston, S. Kravec,… find our card
29. E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving, “Red Teaming Language Models with Language Models,” Feb. 2022. find our card
30. H. Khlaaf, P. Mishkin, J. Achiam, G. Krueger, and M. Brundage, “A Hazard Analysis Framework for Code Synthesis Large Language Models,” July 2022. find
31. J. Maynez, S. Narayan, B. Bohnet, and R. McDonald, “On Faithfulness and Factuality in Abstractive Summarization,” May 2020. find
32. S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring How Models Mimic Human Falsehoods,” May 2022. find our card
33. J. A. Goldstein, G. Sastry, M. Musser, R. DiResta, M. Gentzel, and K. Sedova, “Forecasting potential misuses of language models for disinformation campaigns and how to reduce risk.” https://openai.com/research/forecasting-misuse, Jan. 2023. ↗
34. O. Evans, O. Cotton-Barratt, L. Finnveden, A. Bales, A. Balwit, P. Wills, L. Righetti, and W. Saunders, “Truthful AI: Developing and governing AI that does not lie,” Oct. 2021. find
35. A. Xu, E. Pathak, E. Wallace, S. Gururangan, M. Sap, and D. Klein, “Detoxifying Language Models Risks Marginalizing Minority Voices,” Apr. 2021. find
36. L. Dixon, J. Li, J. Sorensen, N. Thain, and L. Vasserman, “Measuring and Mitigating Unintended Bias in Text Classification,” in Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’18, (New York, NY, USA), pp. 67–73, Association for Computing Machinery, Dec. 2018. find
37. T. Markov, C. Zhang, S. Agarwal, T. Eloundou, T. Lee, S. Adler, A. Jiang, and L. Weng, “A Holistic Approach to Undesired Content Detection in the Real World,” Feb. 2023. find our card
38. OpenAI, “How should AI systems behave, and who should decide?.” https://ope-nai.com/blog/how-should-ai-systems-behave, Feb. 2023. ↗
39. M. Rauh, J. Mellor, J. Uesato, P.-S. Huang, J. Welbl, L. Weidinger, S. Dathathri, A. Glaese, G. Irving, I. Gabriel, W. Isaac, and L. A. Hendricks, “Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models,” Oct. 2022. find
40. S. L. Blodgett, S. Barocas, H. Daumé III, and H. Wallach, “Language (Technology) is Power: A Critical Survey of "Bias" in NLP.” https://arxiv.org/abs/2005.14050v2, May 2020. ↗ our card
41. S. Dev, E. Sheng, J. Zhao, A. Amstutz, J. Sun, Y. Hou, M. Sanseverino, J. Kim, A. Nishi, N. Peng, and K.-W. Chang, “On Measures of Biases and Harms in NLP,” in Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, (Online only), pp. 246–267, Association for Computational Lingu… find
42. T. Bolukbasi, K.-W. Chang, J. Zou, V. Saligrama, and A. Kalai, “Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings,” July 2016. find our card
43. H. Gonen and Y. Goldberg, “Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (L… find our card
44. K. Webster, M. Recasens, V. Axelrod, and J. Baldridge, “Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns,” Oct. 2018. find our card
45. E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? ,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, (Virtual Event Canada), pp. 610–623, ACM, Mar. 2021. find
46. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. Chatterji, A. Chen, K. Creel, J. Q. Davis, D. Demszky, C. Donahue, M. Doumbouya, E. Durmus, S. Ermon, J. Etchemendy, K. Etha… find our card
47. S. U. Noble, Algorithms of Oppression. NYU Press, Feb. 2018. find
48. R. Richardson, J. Schultz, and K. Crawford, “Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice,” Feb. 2019. find
49. W. MacAskill, What We Owe The Future. Basic Books, Aug. 2022. find
50. OpenAI, “GPT-2: 1.5B release.” https://openai.com/research/gpt-2-1-5b-release, Nov. 2019. ↗
51. S. Kreps, R. M. McCain, and M. Brundage, “All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation,” Journal of Experimental Political Science, vol. 9, no. 1, pp. 104–117, 2022/ed. find
52. B. Buchanan, A. Lohn, M. Musser, and K. Sedova, “Truth, Lies, and Automation,” tech. rep., Center for Security and Emerging Technology, May 2021. find
53. A. Myers, “AI’s Powers of Political Persuasion.” https://hai.stanford.edu/news/ais-powers-political-persuasion, Feb. 2023. ↗
54. H. Bai, J. Voelkel, J. Eichstaedt, and R. Willer, “Artificial intelligence can persuade humans on political issues,” 2023. find
55. E. Horvitz, “On the Horizon: Interactive and Compositional Deepfakes,” in INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, pp. 653–661, Nov. 2022. find
56. R. Chesney and D. K. Citron, “Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security,” July 2018. find
57. U.S. Department of Commerce, “Dual use export licenses,” March 13 2023. accessed 2023-03-13. find
58. NATO, “Arms control, disarmament and non-proliferation in nato,” February 27 2023. accessed 2023-02-27. find
59. N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, and C. Raffel, “Extracting Training Data from Large Language Models,” June 2021. find our card
60. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang, “Quantifying Memorization Across Neural Language Models,” Mar. 2023. find
61. D. Ganguli, D. Hernandez, L. Lovitt, N. DasSarma, T. Henighan, A. Jones, N. Joseph, J. Kernion, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, D. Drain, N. Elhage, S. E. Showk, S. Fort, Z. Hatfield-Dodds, S. Johnston, S. Kravec, N. Nanda, K. Ndousse, C. Olsson, D. Amodei, D. Amodei, T. Brown, J. K… find
62. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus, “Emergent Abilities of Large Language Models,” Oct. 2022. find our card
63. R. Ngo, L. Chan, and S. Mindermann, “The alignment problem from a deep learning perspective,” Feb. 2023. find
64. N. Bostrom, Superintelligence: Paths, Dangers, Strategies. United Kingdom: Oxford University Press, Sept. 2014. find
65. A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y. Duan, M. Carroll, M. Lin, A. Mayhew, K. Collins, M. Molamohammadi, J. Burden, W. Zhao, S. Rismani, K. Voudouris, U. Bhatt, A. Weller, D. Krueger, and T. Maharaj, “Harms from Increasingly Agentic Algor… find
66. J. Andreas, “Language Models as Agent Models,” Dec. 2022. find
67. J. Steinhardt, “Emergent Deception and Emergent Optimization.” https://bounded-regret.ghost.io/emergent-deception-optimization/, Feb. 2023. ↗
68. S. M. Omohundro, “The Basic AI Drives,” in Proceedings of the 2008 Conference on Artificial General Intelligence 2008, (NLD), pp. 483–492, IOS Press, June 2008. find
69. N. Bostrom, “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents,” Minds and Machines, vol. 22, pp. 71–85, May 2012. find
70. A. M. Turner, L. Smith, R. Shah, A. Critch, and P. Tadepalli, “Optimal Policies Tend to Seek Power,” Jan. 2023. find
71. A. M. Turner and P. Tadepalli, “Parametrically Retargetable Decision-Makers Tend To Seek Power,” Oct. 2022. find
72. V. Krakovna and janos, “Power-seeking can be probable and predictive for trained agents,” Mar. 2023. find
73. S. Russell, Human Compatible: Artificial Intelligence and the Problem of Control. Cham: Springer International Publishing, 2022. find
74. J. Carlsmith, “Is Power-Seeking AI an Existential Risk?,” June 2022. find
75. Alignment Research Center, “Update on arc’s recent eval efforts,” March 2023 2023. accessed 2023-03-17. find
76. E. Karpas, O. Abend, Y. Belinkov, B. Lenz, O. Lieber, N. Ratner, Y. Shoham, H. Bata, Y. Levine, K. Leyton-Brown, D. Muhlgay, N. Rozen, E. Schwartz, G. Shachaf, S. Shalev-Shwartz, A. Shashua, and M. Tenenholtz, “MRKL Systems: A modular, neuro-symbolic architecture that combines large language models,… find
77. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language Models Can Teach Themselves to Use Tools,” Feb. 2023. find our card
78. G. Mialon, R. Dessì, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Rozière, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, E. Grave, Y. LeCun, and T. Scialom, “Augmented Language Models: A Survey,” Feb. 2023. find our card
79. A. Parisi, Y. Zhao, and N. Fiedel, “TALM: Tool Augmented Language Models,” May 2022. find our card
80. D. Weininger, “Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules,” Journal of chemical information and computer sciences, vol. 28, no. 1, pp. 31–36, 1988. find
81. E. Calvano, G. Calzolari, V. Denicolò, and S. Pastorello, “Artificial Intelligence, Algorithmic Pricing and Collusion,” Apr. 2019. find
82. D. Krueger, T. Maharaj, and J. Leike, “Hidden Incentives for Auto-Induced Distributional Shift,” Sept. 2020. find
83. S. J. DeCanio, “Robots and humans – complements or substitutes?,” Journal of Macroeconomics, vol. 49, pp. 280–291, Sept. 2016. find
84. A. Korinek and J. E. Stiglitz, “Artificial Intelligence and Its Implications for Income Distribution and Unemployment,” in The Economics of Artificial Intelligence: An Agenda, pp. 349–390, University of Chicago Press, Jan. 2018. find
85. J. H. Choi, K. E. Hickman, A. Monahan, and D. Schwarcz, “ChatGPT Goes to Law School,” Jan. 2023. find
86. L. R. Raymond, E. Brynjolfsson, and D. Li, “Augmented intelligence: The effects of ai on productivity and work practices,” Sep 2022. find
87. E. van Inwegen, Z. Munyikwa, and J. J. Horton, “Algorithmic Writing Assistance on Jobseekers’ Resumes Increases Hires,” Jan. 2023. find
88. A. Ziegler, E. Kalliamvakou, S. Simister, G. Sittampalam, A. Li, A. Rice, D. Rifkin, and E. Aftandilian, “Productivity Assessment of Neural Code Completion,” May 2022. find
89. S. Noy and W. Zhang, “Experimental evidence on the productivity effects of generative artificial intelligence,” Available at SSRN 4375283, 2023. find
90. S. Peng, E. Kalliamvakou, P. Cihon, and M. Demirer, “The impact of ai on developer productivity: Evidence from github copilot,” arXiv preprint arXiv:2302.06590, 2023. — [arXiv:2302.06590](https://arxiv.org/abs/2302.06590) find
91. D. Acemoglu and P. Restrepo, “Demographics and Automation,” The Review of Economic Studies, vol. 89, pp. 1–44, Jan. 2022. find
92. Partnership on AI, “AI and Job Quality,” tech. rep., Partnership on AI, Sept. 2022. find
93. “OpenAI Charter.” https://openai.com/charter, Apr. 2018. ↗
94. S. Armstrong, N. Bostrom, and C. Shulman, “Racing to the precipice: A model of artificial intelligence development,” Technical 2013-1, Future of Humanity Institute, Oct. 2013. find
95. P. E. Tetlock and D. Gardner, Superforecasting: The Art and Science of Prediction. Crown, Sept. 2015. find
96. S. Passi and M. Vorvoreanu, “Overreliance on AI Literature Review,” tech. rep., AI Ethics and Effects in Engineering and Research, June 2022. find
97. PAI, “Data enrichment sourcing guidelines,” November 2022 2022. accessed 2023-03-13. find
98. PAI, “Responsible sourcing of data enrichment services,” June 2021 2021. accessed 2023-03-13. find
99. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” Aug. 2017. find
100. A. Glaese, N. McAleese, M. Trębacz, J. Aslanides, V. Firoiu, T. Ewalds, M. Rauh, L. Weidinger, M. Chadwick, P. Thacker, L. Campbell-Gillingham, J. Uesato, P.-S. Huang, R. Comanescu, F. Yang, A. See, S. Dathathri, R. Greig, C. Chen, D. Fritz, J. S. Elias, R. Green, S. Mokrá, N. Fernando, B. Wu, R. Fo… find our card
101. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, C. Chen, C. Olsson, C. Olah, D. Hernandez, D. Drain, D. Ganguli, D. Li, E. Tran-Johnson, E. Perez, J. Kerr, J. Mueller, J. Ladish, J. Landau, K. Ndousse, K. Lukosuite, L. Lovitt, M. Sellit… find our card
102. S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith, “RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models,” Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3356–3369, 2020. find our card
103. OpenAI, “Introducing chatgpt,” November 2022 2020. accessed 2023-03-13. find
104. OpenAI, “Openai api,” June 2020 2020. accessed 2023-03-13. find
105. T. Davidson, D. Bhattacharya, and I. Weber, “Racial Bias in Hate Speech and Abusive Language Detection Datasets,” in Proceedings of the Third Workshop on Abusive Language Online, (Florence, Italy), pp. 25–35, Association for Computational Linguistics, Aug. 2019. find our card