Elamur Research
☰
home / papers / 2023 / AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation

AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2308.08155

Summary

As LLM-based agents take on more complex work, there is a need for a general way to build applications where several agents cooperate. Such agents might be backed by models, people, or software tools. AutoGen, from Microsoft Research and academic collaborators, is an open-source framework built on that idea. Its first concept is the conversable agent: a reusable component that exchanges messages, keeps its own context, and draws on any mix of LLM reasoning, human input, and code or function execution. Its second concept is conversation programming. Developers define the agents and then specify how they interact, mixing plain-language prompts with Python control logic. This supports static or dynamic, two-agent or group, and fully automated or human-in-the-loop setups. The authors evaluate the framework on tasks ranging from math problem solving and software programming to answering questions, tackling operations-research problems, making decisions in interactive online environments, and entertainment. They claim better results than prior state-of-the-art approaches, less development code, and lower manual effort. The work matters because it offers a single modular abstraction for multi-agent LLM systems. It also highlights open issues around safety, accountability, and the right balance of automation versus human control.

Key points

  • The built-in agent hierarchy has ConversableAgent as its top-level abstraction. Two preconfigured subclasses sit beneath it: AssistantAgent (LLM-backed, no human input by default) and UserProxyAgent (requests human input and runs code or function calls). A GroupChatManager coordinates multi-agent chats.
  • All agents share a unified interface built on send, receive, and generate_reply. Developers can attach custom reply functions with register_reply; otherwise, default auto-reply functions are used.
  • The human_input_mode setting (e.g., "NEVER" or "ALWAYS") controls how often humans take part.
  • The enhanced LLM inference layer adds result caching, error handling, and message templating.
  • The code is available at github.com/microsoft/autogen (arXiv:2308.08155). The ethics notes flag privacy, bias, transparency, and the risk of agents changing their environment, such as by installing packages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

394 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 380 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.08155 · arXiv 2308.08155

@misc{wu2023autogen,
  title = {{AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation}},
  author = {Wu, Qingyun and Bansal, Gagan and Zhang, Jieyu and Wu, Yiran and Li, Beibin and Zhu, Erkang and Jiang, Li and Zhang, Xiaoyun and Zhang, Shaokun and Liu, Jiale and Awadallah, Ahmed Hassan and White, Ryen W and Burger, Doug and Wang, Chi},
  year = {2023},
  doi = {10.48550/arxiv.2308.08155},
  eprint = {2308.08155},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.08155}
}

References

46 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy. Evaluating correctness and faithfulness of instruction-following models for question answering. arXiv preprint arXiv:2307.16877, 2023. find
  2. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Col-lisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. Guidelines for human-ai interaction. In Proceedings of the 2019 chi conference on human factors in computing systems, 2019. find
  3. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety, 2016. find our card
  4. AutoGPT. Documentation — auto-gpt. https://docs.agpt.co/, 2023. ↗
  5. BabyAGI. Github — babyagi. https://github.com/yoheinakajima/babyagi, 2023. ↗
  6. Carrie J. Cai, Samantha Winter, David F. Steiner, Lauren Wilcox, and Michael Terry. ”hello ai”: Uncovering the onboarding needs of medical practitioners for human-ai collaborative decision-making. Proceedings of the ACM on Human-Computer Interaction, 2019. find
  7. Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint arXiv:2305.17126, 2023. find
  8. Chroma. Chromadb. https://github.com/chroma-core/chroma, 2023. ↗
  9. Victor Dibia. LIDA: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Toronto, Canada, July 2023. Association f… find
  10. Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. arXiv preprint arXiv:2304.07590, 2023. find
  11. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325, 2023. find our card
  12. Atty Eleti, Jeff Harris, and Logan Kilpatrick. Function calling and other api updates. https: //openai.com/blog/function-calling-and-other-api-updates, 2023. find
  13. Guidance. Guidance. https://github.com/guidance-ai/guidance, 2023. ↗
  14. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. find our card
  15. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023. find our card
  16. Eric Horvitz. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 1999. find
  17. HuggingFace. Transformers agent. transformers\_agents, 2023. https://huggingface.co/docs/transformers/ ↗
  18. Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks. arXiv preprint arXiv:2303.17491, 2023. find our card
  19. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 201… find our card
  20. LangChain. Introduction — langchain. https://python.langchain.com/en/latest/index. html, 2023. ↗
  21. Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra. Deal or no deal? end-to-end learning for negotiation dialogues. arXiv preprint arXiv:1706.05125, 2017. — [arXiv:1706.05125](https://arxiv.org/abs/1706.05125) find
  22. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 2020. find
  23. Beibin Li, Konstantina Mellou, Bo Zhang, Jeevan Pathuri, and Ishai Menache. Large language models for supply chain optimization. arXiv preprint arXiv:2307.03875, 2023a. — [arXiv:2307.03875](https://arxiv.org/abs/2307.03875) find
  24. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for ”mind” exploration of large scale language model society, 2023b. find
  25. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. Encouraging divergent thinking in large language models through multi-agent debate, 2023. find our card
  26. Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. arXiv preprint arXiv:1802.08802, 2018. find
  27. Jerry Liu. LlamaIndex, November 2022. URL https://github.com/jerryjliu/llama\_index. ↗
  28. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. find
  29. Roberto Navigli, Simone Conia, and Björn Ross. Biases in large language models: Origins, inventory and discussion. ACM Journal of Data and Information Quality, 2023. find
  30. OpenAI. ChatGPT plugins. https://openai.com/blog/chatgpt-plugins, 2023. ↗
  31. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023. find
  32. Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. Retrieval augmented code generation and summarization. arXiv preprint arXiv:2108.11601, 2021. find
  33. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023. find our card
  34. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084. ↗
  35. Semantic kernel. https://github.com/microsoft/semantic-kernel, Bokui Shen, Fei Xia, Chengshu Li, Roberto Martı́n-Martı́n, Linxi Fan, Guanzhi Wang, Claudia Pérez-D’Arpino, Shyamal Buch, Sanjana Srivastava, Lyne Tchapmi, et al. igibson 1.0: A simulation environment for interactive tasks in large real… ↗
  36. Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of bits: An open-domain platform for web-based agents. In International Conference on Machine Learning. PMLR, 2017. find
  37. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/201… ↗ our card
  38. Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, et al. Starcraft ii: A new challenge for reinforcement learning. arXiv preprint arXiv:1708.04782, 2017. — [arXiv:1708.04782](https://arxiv.org/abs/1708.04782) find
  39. Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. Flaml: A fast and lightweight automl library. Proceedings of Machine Learning and Systems, 2021. find
  40. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. find
  41. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432, 2023b. find our card
  42. Daniel S. Weld and Oren Etzioni. The first law of robotics (a call to arms). In AAAI Conference on Artificial Intelligence, 1994. find
  43. Max Woolf. Langchain problem. https://minimaxir.com/2023/07/langchain-problem/, 2023. ↗
  44. Yiran Wu, Feiran Jia, Shaokun Zhang, Qingyun Wu, Hangyu Li, Erkang Zhu, Yue Wang, Yin Tat Lee, Richard Peng, and Chi Wang. An empirical study on challenging math problem solving with gpt-4. arXiv preprint arXiv:2306.01337, 2023. find
  45. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864, 2023. find our card
  46. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. find our card