从基础智能到通用智能: 基于大模型的GenAI和AGI之现状与展望
Summary
This survey and position paper, led by researchers at the CAS Institute of Automation, reviews the state of generative AI (GenAI) worldwide and in China. It asks how China can close its gap with the United States on the road toward artificial general intelligence (AGI).
The authors trace the technical lineage from deep learning and the Transformer to diffusion models. They catalogue leading foreign systems, including GPT-4, Claude 3, Llama 2, Stable Diffusion and Sora. They then map China's crowded field of general models (文心, 通义千问/Qwen, 盘古, Kimi, InternLM) and domain models in medicine, finance, education, industry and AI4S. A side-by-side comparison of compute, data, algorithms and ecosystem shows a clear US lead. The paper also contrasts Western and Chinese approaches to AI safety governance across the model lifecycle.
As remedies, it proposes:
- energy-efficient compute powered by western China's renewables;
- federated data;
- specialised domain models;
- a blockchain/DAO-based federated ecosystem called TAO.
It ends with a speculative vision of a society shared by digital humans, robots and people. Its main value is as a map of China's LLM landscape and policy thinking.
Key points
- Hardware gap: Nvidia H100 (SXM) delivers 1,979 TFLOPS FP16 with 80 GB HBM3 and 900 GB/s interconnect. Huawei Ascend 910 delivers 280 TFLOPS with 16 GB. The software stacks also differ: CUDA versus CANN.
- Ecosystem gap: China has 70 AI unicorns worth about 1.3T, against 315 worth 5.9T in the US. China has 60+ highly cited papers versus 220+, and its share of institutions/talent is about 14%/13.85% versus 55%/56.55%. Sources are State of AI Report 2023 and Aminer.
- Notable Chinese models:
- Kimi's context window grew from 200k to 2 million Chinese characters.
- Pangu weather model, trained on 39 years of global data, reportedly beat numerical forecasting.
- 紫东太初 was billed as the first vision–text–speech pretrained model.
- Domestic governance: It follows the 《生成式人工智能服务管理暂行办法》, which introduced a filing ("备案") regime. Its seven content and compliance dimensions are checked through red-team tests, sandboxes and third-party evaluation such as OpenCompass, C-Eval and SuperCLUE.
- Federated data and TAO: The proposed federated data architecture has six components: real data, virtual data, federated experiments, fusion, security and trusted federated intelligence. TAO combines blockchain "TRUE" with DAO. The envisioned future workforce is over 80% digital humans, under 15% robots and under 5% biological humans.
Cite
@inproceedings{2024,
title = {{从基础智能到通用智能: 基于大模型的GenAI和AGI之现状与展望}},
author = {缪青海 and 王兴霞 and 杨静 and 赵勇 and 王雨桐 and 陈圆圆 and 田永林 and 俞怡 and 林懿伦 and 鄢然 and 马嘉琪 and 那晓翔 and 王飞跃},
booktitle = {自动化学报 (Acta Automatica Sinica)},
year = {2024},
doi = {10.16383/j.aas.c240156},
url = {https://doi.org/10.16383/j.aas.c240156}
}References
- Wang F Y, Miao Q H, Li X, Wang X X, Lin Y L. What does ChatGPT say: The DAO from algorithmic intelligence to linguistic intelligence. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575−579 Wang F Y, Miao Q H, Li L X, Ni Q H, Li X, Li J J, et al. When does sora show: The beginning of TAO to imag… — [doi:10.1109/tiv.2024.3376575](https://doi.org/10.1109/tiv.2024.3376575) find
- Nature, 2015, 521(7553): 436−444 LeCun Y, Boser B, Denker J S, Henderson D, Howard R E, Hubbard W, et al. Backpropagation applied to handwritten zip code recognition. find
- Neural Computation, 1989, 1(4): 541−551 Graves A, Schmidhuber J. Supervised sequence labelling with recurrent neural networks. In: Proceedings of the Conference and Workshop on Neural Information Processing Systems. 2009. Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief net… find
- Neural Computation, 2006, 18(7): 1527− 1554 Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. find
- Lake Tahoe, USA: Curran Associates Inc., 2012. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, et al. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. find
- Long Beach, USA: Curran Associates Inc., 2017. 5998−6008 Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human… find
- Minneapolis, USA: ACL, 2019. 4171−4186 Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Language Understanding by Generative Pre-Training, OpenAI Technical Report, 2018. Dai Z H, Yang Z L, Yang Y M, Carbonell J, Le Q, Salakhutdinov R. Transformer-XL: Attentive language models beyond a fix… find
- Florence, Italy: ACL, 2019. 2978−2988 Du H P, Teng S Y, Chen H, Ma J Q, Wang X, Gou C, et al. Chat with ChatGPT on intelligent vehicles: An IEEE TIV perspective. IEEE Transactions on Intelligent Vehicles, 2023, 8(3): 2020−2026 Gao Y B, Tong W, Wu E Q, Chen W, Zhu G Y, Wang F Y. Chat with ChatGPT on… find
- Acta Automatica Sinica, 2023, 49(4): 705−717 (卢经纬, 郭超, 戴星原, 缪青海, 王兴霞, 杨静, 等. 问答 ChatG-PT 之后: 超大预训练模型的机遇和挑战. 自动化学报, 2023, 49(4): 705−717) Tian Yong-Lin, Wang Yu-Tong, Wang Jian-Gong, Wang Xiao, Wang Fei-Yue. Key problems and progress of vision transformers: The state of the art and prospects. find
- Acta Automatica Sinica, 2022, 48(4): 957−979 (田永林, 王雨桐, 王建功, 王晓, 王飞跃. 视觉 Transformer 研究 的关键问题: 现状及展望. 自动化学报, 2022, 48(4): 957−979) Yang Z L, Dai Z H, Yang Y M, Carbonell J, Salakhutdinov Le Q V. XLNet: Generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd Int… find
- Vancouver, Canada: Curran Associates Inc., 2019. Article No. 517 Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. find
- Vancouver, Canada: Curran Associates Inc., 2020. Article No. 574 Liu Y H, Ott M, Goyal N, Du J F, Joshi M, Chen D Q, et al. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv: 1907.11692, 2019. Lewis M, Liu Y H, Goyal N, Ghazvininejad M, Mohamed A, Levy O, et al. BART: Den… find
- ACL, 2020. 7871− 7880 Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 2020, 21(1): Article No. 140 Radford A, Narasimhan K, Salimans T, Sutskever I. Improvin… find
- Open-AI, 2018, 8(9−10): 1−9 Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Florencia B, et al. GPT-4 technical report. arXiv preprint arXiv: 2303. 08774, 2023. find
- Anil R, Dai A M, Firat O, Johnson M, Lepikhin D, Passos A, et al. PaLM 2 technical report. arXiv preprint arXiv: 2305. 10403, 2023. find our card
- Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M A, Lacroix T, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv: 2302.13971, 2023. find
- Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv: 2307.09288, 2023. find
- Anthropic. Introducing the next generation of Claude [Online], available: https://www.anthropic.com/news/claude-3-family, March 28, 2024 ↗
- Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, et al. An image is worth 16×16 words: Transformers for image recognition at scale. In: Proceedings of the International Conference on Learning Representations. 2021. find
- Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M. Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv: 2204.06125, 2022. find our card
- Betker J, Goh G. Improving image generation with better captions. Computer Science, 2023, 2(3): 8 find
- Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 10674−10685 find
- Singer U, Polyak A, Hayes T, Yin X, An J, Zhang S Y, et al. Make-A-Video: Text-to-video generation without text-video data. arXiv preprint arXiv: 2209.14792, 2022. — [arXiv:2209.14792](https://arxiv.org/abs/2209.14792) find
- Bar-Tal O, Chefer H, Tov O, Herrmann C, Paiss R, Zada S, et al. Lumiere: A space-time diffusion model for video generation. arXiv preprint arXiv: 2401.12945, 2024. — [arXiv:2401.12945](https://arxiv.org/abs/2401.12945) find
- Brooks T, Peebles B, Holmes C, DePue W, Guo Y F, Jing L, et al. Video generation models as world simulators [Online], available: https://openai.com/research/video-generation-mode-ls-as-world-simulators, March 28, 2024 ↗
- Sun Y, Wang S H, Feng S K, Ding S Y, Pang C, Shang J Y, et al. ERNIE 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv: 2107.02137, 2021. find our card
- Bai J Z, Bai S, Chu Y F, Cui Z Y, Dang K, Deng X D, et al. Qwen technical report. arXiv preprint arXiv: 2309.16609, 2023. find our card
- Zeng W, Ren X Z, Su T, Wang H, Liao Y, Wang Z W, et al. PanGu- α : Large-scale autoregressive pretrained Chinese language models with auto-parallel computation. arXiv preprint arXiv: 2104.12369, 2021. find
- Team I L M. InternLM: A multilingual language model with progressively enhanced capabilities [Online], available: https://github.com/InternLM/InternLM, March 28, 2024 ↗
- Yang A Y, Xiao B, Wang B N, Zhang B R, Bian C, Yin C, et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv: 2309.10305, 2023. find our card
- Wang Fei-Yue. Digital teachers and parallel education: A paradigm shift in teaching and learning after ChatGPT. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 454−463 find
- (王飞跃. 数字教师与平行教育: 关于 ChatGPT 之后教学变革的 探讨. 智能科学与技术学报, 2023, 5(4): 454−463) find
- Wang Fei-Yue. Parallel doctors and parallel hospitals: Impact and perspective of ChatGPT-like AIGC and AGI on medicine and medicare. Medical Journal of Peking Union Medical College Hospital, 2023, 14(4): 673−679 find
- (王飞跃. 平行医生与平行医院: ChatGPT 与通用人工智能技术 对未来医疗的冲击与展望. 协和医学杂志, 2023, 14(4): 673−679) find
- Lu J W, Wang X X, Cheng X, Yang J, Kwan O, Wang X. Parallel factories for smart industrial operations: From big AI models to field foundational models and scenarios engineering. IEEE/CAA Journal of Automatica Sinica, 2022, 9(12): 2079− 2086 find
- Yang J, Li S M, Wang X X, Lu J W, Wu H Y, Wang X. De-FACT in ManuVerse for parallel manufacturing: Foundation models and parallel workers in smart factories. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2188−2199 Yang J, Wang Y T, Wang X X, Wang X X, Wang X, Wang F Y. Ge… — [doi:10.1109/tsmc.2024.3349555](https://doi.org/10.1109/tsmc.2024.3349555) find
- IEEE Intelligent Systems, 2022, 37(6): 77−82 Wang X, Wang Y T, Yang J, Jia X F, Li L J, Ding W P, et al. The survey on multi-source data fusion in cyber-physical-social systems: Foundational infrastructure for industrial metaverses and industries 5.0. find
- Information Fusion, 2024, 107: Article No. 102321 Wang X X, Cheng X, Lu J W, Kwan O, Li S X, Ping Z X. Metaverses-based parallel oil fields in CPSS: A framework and methodology. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2138−2147 Wang Y T, Wang J G, Cao Y S, Li S X, K… find
- IEEE, 2023. 4289−4294 Wang J G, Tian Y L, Wang Y T, Yang J, Wang X X, Wang S J, et al. A framework and operational procedures for metaverses-based industrial foundation models. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2037−2046 Bi K F, Xie L X, Zhang H H, Chen X, Gu… find
- Nature, 2023, 619(7970): 533−538 Dan Y H, Lei Z K, Gu Y Y, Li Y, Yin J H, Lin J J, et al. EduChat: A large-scale language model-based chatbot system for intelligent education. arXiv preprint arXiv: 2308.02773, 2023. Chen Y R, Wang Z Y, Xing X F, Zheng H M, Xu Z P, Fang K, et al. BianQue-1.0: Improvi… find
- Singapore: ACL, 2023. 1170− 1183 Zhang H B, Chen J Y, Jiang F, Yu F, Chen Z H, Chen G M, et al. HuatuoGPT, towards taming language model to be a doctor. In: Proceedings of the Findings of the Association for Computational Linguistics. find
- Zhang X Y, Yang Q. XuanYuan 2.0: A large Chinese financial chat model with hundreds of billions parameters. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. Birmingham, UK: ACM, 2023. 4435−4439 find
- OpenAI. Our approach to alignment research [Online], available: https://openai.com/blog/our-approach-to-alignment-rese-arch, March 28, 2024 ↗
- Center for Research on Foundation Models. A holistic framework for evaluating foundation models [Online], available: https://crfm.stanford.edu/helm/lite/latest/\#/, March 28, 2024 ↗
- European Commission. Excellence and trust in artificial intelligence [Online], available: https://commission.europa.eu/strate-gy-and-policy/priorities-2019-2024/europe-fit-digital-age/excel-lence-and-trust-artificial-intelligence_en, March 28, 2024 ↗
- Liu A W, Pan L Y, Lu Y J, Li J J, Hu X M, Zhang X, et al. A survey of text watermarking in the era of large language models. arXiv preprint arXiv: 2312.07913, 2024. — [arXiv:2312.07913](https://arxiv.org/abs/2312.07913) find
- Dai J, Pan X H, Sun R Y, Ji J M, Xu X B, Liu M, et al. Safe RLHF: Safe reinforcement learning from human feedback. arXiv preprint arXiv: 2310.12773, 2023. find our card
- Zhang W E, Sheng Q Z, Alhazmi A, Li C L. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology, 2020, 11(3): Article No. 24 find
- Office of the Central Cyberspace Affairs Commission. Global AI governance initiative [Online], available: https://www.cac.gov.cn/2023-10/18/c_1699291032884978.htm, March 28, 2024 ↗
- (中央网络安全和信息化委员会办公室. 全球人工智能治理倡议 [Online], available: https://www.cac.gov.cn/2023-10/18/c_1699 291032884978.htm, 2024-03-28) ↗
- National Internet Information Office, National Development and Reform Commission, Ministry of Education of the People' s Republic of China, Ministry of Science and Technology of the People' s Republic of China, Ministry of Industry and Information Technology of the People' s Republic of China, Minis… find
- (国家互联网信息办公室, 中华人民共和国国家发展和改革委员 会, 中华人民共和国教育部, 中华人民共和国科学技术部, 中华人 民共和国工业和信息化部, 中华人民共和国公安部, 等. 生成式人 工智能服务管理暂行办法 [Online], available: https://www.cac.gov.cn/2023-07/13/c_1690898327029107.htm, 2024-03-28) ↗
- Yu Y, Yao S Y, Li J J, Wang F Y, Lin Y L. SWDPM: A social welfare-optimized data pricing mechanism. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC). Honolulu, USA: IEEE, 2023. 2900−2906 find
- Wang D, Wang X H, Chen L, Yao S Y, Jing M, Li H H, et al. TransWorldNG: Traffic simulation via foundation model. In: Proceedings of the IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). Bilbao, Spain: IEEE, 2023. 6007−6012 find
- Yu J R, Yu Y, Yao S Y, Wang D, Cai P L, Li H H, et al. RoW-based parallel control for mixed traffic scenario: A case study on lane-changing. In: Proceedings of the IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). find
- Bilbao, Spain: IEEE, 2023. 5397−5402 find
- Yao S Y, Yu J R, Yu Y, Xu J, Dai X Y, Li H H, et al. Towards integrated traffic control with operating decentralized autonomous organization. In: Proceedings of the IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). Bilbao, Spain: IEEE, 2023. 6126−6131 find
- Qi H S, Yu Y, Tang Q, Hu X B. Intersection traffic deadlock formation and its probability: A petri net-based modeling approach. IET Intelligent Transport Systems, 2022, 16(10): 1342− 1363 find
- Lin Y L, Hu W, Chen X, Li S, Wang F Y. City 5.0: Towards spatial symbiotic intelligence via DAOs and parallel systems. IEEE Transactions on Intelligent Vehicles, 2023, 8(7): 3767− 3770 find
- Yu Y, Yao S Y, Wang K X, Chen Y, Wang F Y, Lin Y L. Pursuing equilibrium of medical resources via data empowerment in parallel healthcare system. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC). Honolulu, USA: IEEE, 2023. 3186−3191 Yu Y, Cui Y L, Zeng J Q,… find
- Physica A: Statistical Mechanics and Its Applications, 2022, 591: Article No. 126750 Lin Y L, Na X X, Wang D, Dai X Y, Wang F Y. Mobility 5.0: Smart logistics and transportation services in cyber-physical-social systems. IEEE Transactions on Intelligent Vehicles, 2023, 8(6): 3527−3532 Xu J, Yao S Y,… find
- Bilbao, Spain: IEEE, 2023. 5391−5396 Li L, Wang X, Wang K F, Lin Y L, Xin J M, Chen L, et al. Parallel testing of vehicle intelligence via virtual-real interaction. find
- Science Robotics, 2019, 4(28): Article No. eaaw4106 Zhao Y, Hu C, Zhu Z Q, Qiu S H, Chen B, Jiao P, et al. Crowd sensing intelligence for ITS: Participants, methods, and stages. IEEE Transactions on Intelligent Vehicles, 2023, 8(6): 3541−3546 Zhang J P, Pu J, Xue J R, Yang M, Xu X, Wang X, et al. Hi… find
- Pattern Recognition and Artificial Intelligence, 2017, 30(8): 673− 681 (刘昕, 王晓, 张卫山, 汪建基, 王飞跃. 平行数据: 从大数据到数 据智能. 模式识别与人工智能, 2017, 30(8): 673−681) Li X, Tian Y L, Ye P J, Duan H B, Wang F Y. A novel scenarios engineering methodology for foundation models in metaverse. IEEE Transactions on Systems, Ma… find
- Acta Automatica Sinica, 2021, 47(8): 1912−1920 (朱静, 王飞跃, 王戈, 田永林, 袁勇, 王晓, 等. 联邦控制: 面向信 息安全和权益保护的分布式控制方法. 自动化学报, 2021, 47(8): 1912−1920) Wang F Y, Qin R, Li J J, Wang X, Qi H W, Jia X F, et al. Federated management: Toward federated services and federated security in federated ecology. IEEE Transacti… find
- The Innovation, 2023, 4(6): Article No. 100521 Yang J, Wang X, Tian Y L, Wang X, Wang F Y. Parallel intelligence in CPSSs: Being, becoming, and believing. find
- IEEE Intelligent Systems, 2023, 38(6): 75−80 Li X, Ye P J, Li J J, Liu Z M, Cao L B, Wang F Y. From features engineering to scenarios engineering for trustworthy AI: I&I, C&C, and V&V. find
- IEEE Intelligent Systems, 2022, 37(4): 18−26 Wang Fei-Yue. Parallel management: The DAO to smart ecological technology for complexity management intelligence. find
- Acta Automatica Sinica, 2022, 48(11): 2655−2669 (王飞跃. 平行管理: 复杂性管理智能的生态科技与智慧管理之 DAO. 自动化学报, 2022, 48(11): 2655−2669) Li J J, Qin R, Wang F Y. The future of management: DAO to smart organizations and intelligent operations. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(6): 3389… find
- Acta Automatica Sinica, 2023, 49(3): 614−634 (杨 静 , 王 晓 , 王 雨 桐 , 刘 忠 民 , 李 小 双 , 王 飞 跃 . 平 行 智 能 与 CPSS: 三 十 年 发 展 的 回 顾 与 展 望 . 自 动 化 学 报 , 2023, 49(3): 614−634) Wang X X, Yang J, Wang Y T, Miao Q H, Wang F Y, Zhao A J, et al. Steps toward industry 5.0: Building “6S” parallel industries with cyber… find
- 缪青海 中国科学院大学人工智能学 院副教授. 主要研究方向为智能系统, 智能交通, 平行智能. E-mail: miaoqh@ucas.ac.cn (MIAO Qing-Hai Associate professor at School of Artificial Intelligence, University of Chinese Academy of Sciences. His research interest covers intelligent systems, intelligent transportation systems, parallel intelligence.)… find
- Her research interest covers parallel intelligence, parallel oilfields, and multi-agent systems.) 杨 静 中国科学院自动化研究所复 杂系统管理与控制国家重点实验室博 士研究生. 2020 年获得北京化工大学 自动化专业学士学位. 主要研究方向 为平行制造, 社会制造, 人工智能和 社会物理信息系统. E-mail: yangjing2020@ia.ac.cn (YANG Jing Ph.D. candidate at the State Key Laboratory for Management… find
- 赵 勇 国防科技大学系统工程学院 博士研究生. 2021 年获国防科技大学 控制科学与工程专业硕士学位. 主要 研究方向为移动群智感知, 空间众包, 人机交互. E-mail: zhaoyong15@nudt.edu.cn (ZHAO Yong Ph.D. candidate at the College of Systems Engineering, National University of Defense Technology. He received his master degree in control science and engineering from National… find
- 王雨桐 中国科学院自动化研究所助 理研究员. 2021 年获得中国科学院大 学控制理论与控制工程专业博士学 位. 主要研究方向为计算机视觉. E-mail: yutong.wang@ia.ac.cn (WANG Yu-Tong Assistant professor at the Institute of Automation, Chinese Academy of Sciences. She received her Ph.D. degree in control theory and control engineering from the University of Chinese Ac… find
This page in Markdown → https://md.research.elamur.ai/free/papers/miao-2024-foundation-intelligence-general/