Elamur Research⌕
☰
home / papers / 2026 / Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design

Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design

Paper · 2026 · Zhilun Zhou, Zihan Liu, Jiahe Liu et al.

Summary

Most work on LLM-based multi-agent systems (MAS) tunes a system for a single task. This paper asks whether agent groups have a task-independent "general ability," similar to the collective intelligence (CI) factor found in human teams. The authors adapt human group-psychology experiments and build 108 LLM agent groups from 8 LLMs. The groups differ in size, number of interaction rounds, communication topology, and whether they mix models or use only one. All groups run a debate-style framework in which agents revise their answers from what their neighbours said, and the answers are then aggregated. Each group is tested on commonsense reasoning, mathematics, games, coding, and writing. Factor analysis finds a dominant latent factor, called Artificial Collective Intelligence (ACI), that predicts how groups perform on new tasks. As in human studies, the collaboration process matters most. The authors then train a model that predicts ACI from group features. Used as a regularizing plug-in for MAS optimization, it improves results on unseen tasks. The work points toward MAS designs that generalize rather than overfit to one task.

Key points

  • Setup: 108 groups were built from 8 models: gpt-3.5-turbo-0125, gpt-4o-mini, Qwen2.5-7B/32B/72B-Instruct, glm-4-9b-chat, internlm2_5-20b-chat, and gemma-2-27b-it.
    • Group sizes were 3, 5, or 8, with 2 or 3 rounds.
    • Topologies were decentralized, centralized, or random.
  • Cross-task consistency: Performance on different tasks correlates at an average of r = 0.55. Human groups show r = 0.28.
  • ACI strength: A single ACI factor accounts for 66.3% of performance variance. The comparable figure for human groups is roughly 43%.
  • What drives ACI: The collaboration process has the largest effect, individual model intelligence comes second, and group size matters least.
  • Practical use: The ACI-prediction plug-in raises MAS optimization performance on unseen tasks by 4.6% on average. Code is available at github.com/tsinghua-fib-lab/LLM_Collective_Intelligence.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Networks and Communications · 100 works

Cite

Cite as Zhilun Zhou, Zihan Liu, Jiahe Liu, Yihan Wang, Qingyu Shao, Fengli Xu, Depeng Jin and Yong Li. 2026. Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.624

@inproceedings{zhou2026identifying,
  title = {{Identifying Collective Intelligence Factor in LLM Agent Groups for Generalizable Multi-Agent System Design}},
  author = {Zhilun Zhou, Zihan Liu, Jiahe Liu, Yihan Wang, Qingyu Shao, Fengli Xu, Depeng Jin and Yong Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.624},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.624}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. with self-feedback. Advances in Neural Information Processing Systems, 36. find
  2. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  3. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, and 1 others. 2024. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap…
  4. Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025. Scaling large language model-based multi-agent collaboration. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Sin…
  5. Christoph Riedl, Young Ji Kim, Pranav Gupta, Thomas W Malone, and Anita Williams Woolley. 2021. Quantifying collective intelligence in human groups. Proceedings of the National Academy of Sciences, 118(21):e2005737118. find
  6. Andries P. Smit, Nathan Grinsztajn, Paul Duckworth, Thomas D. Barrett, and Arnu Pretorius. 2024. Should we be going mad? A look at multi-agent debate strategies for llms. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net. find
  7. Charles Spearman. 1904. “general intelligence,” objectively determined and measured. The American Journal of Psychology, 15(2):201–293. find
  8. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, and 1 others. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Le…
  9. Samuel PL Veissière, Axel Constant, Maxwell JD Ram-stead, Karl J Friston, and Laurence J Kirmayer. 2020. Thinking through other minds: A variational approach to cognition and culture. Behavioral and brain sciences, 43:e90. find
  10. Junlin Wang, Jue WANG, Ben Athiwaratkun, Ce Zhang, and James Zou. 2025. Mixture-of-agents enhances large language model capabilities. In The Thirteenth International Conference on Learning Representations.
  11. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark…
  12. Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of ‘small-world’networks. nature, 393(6684):440–442. find
  13. Anita Williams Woolley, Christopher F Chabris, Alex Pentland, Nada Hashmi, and Thomas W Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. science, 330(6004):686–688. find
  14. Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, and 1 others. 2023. Chain-of-experts: When llms meet complex operations research problems. In The Twelfth International Conference on Learning Representations. find
  15. Miao Yu, Shilong Wang, Guibin Zhang, Junyuan Mao, Chenlong Yin, Qijiong Liu, Kun Wang, Qingsong Wen, and Yang Wang. 2025. Netsafe: Exploring the topological safety of multi-agent system. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025… find
  16. Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jeffrey Xu Yu, and Tianlong Chen. 2025a. Cut the crap: An economical communication pipeline for llm-based multi-agent systems. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Sin… find
  17. Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, Tianlong Chen, and Dawei Cheng. 2025b. G-designer: Architecting multi-agent communication topologies via graph neural networks. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver,… find
  18. Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2024. Exploring collaboration mechanisms for LLM agents: A social psychology view. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thail…
  19. Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulić, Anna Korhonen, and Sercan Ö Arık. 2025. Multi-agent design: Optimizing agents with better prompts and topologies. arXiv preprint arXiv:2502.02533. — [arXiv:2502.02533](https://arxiv.org/abs/2502.02533) find
  20. Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. 2024. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-identifying-collective-intelligence-factor-llm/