面向边缘智能的大模型研究进展
Summary
This survey examines how large models (LLMs, multimodal and vision models such as LLaMA, LLaVA and SAM) can be moved from costly cloud infrastructure toward edge devices. Cloud-only training and serving are expensive, scale poorly and expose user data. Running models closer to where data originates can cut latency and bandwidth use and improve privacy. The authors organize the literature along two axes.
For training, the survey focuses on fine-tuning. It argues that parameter-efficient methods combined with federated learning are the most practical route. The covered methods are adapters, prefix tuning, prompt tuning and LoRA. It also compares existing federated fine-tuning frameworks on two criteria: privacy protection and support for heterogeneous clients.
For inference, it groups techniques into three stages: compression before deployment, model-level optimization after deployment, and system-level optimization after deployment. It also reviews complete on-device deployment toolkits. Hardware acceleration is deliberately left out.
The paper ends with open challenges and future directions. Its value lies in offering a structured map of a fast-growing field for both researchers and industry.
Key points
- Motivating costs:
- Pretraining Llama2-70B took about 1.72 million GPU hours on 760 A100 nodes.
- Half-precision inference for that model needs at least 140 GB of memory.
- A single GPT-3 (175B) training run cost roughly US$4.6M on 499B tokens.
- Training costs are projected to reach about $1B by 2027.
- Federated PEFT workflow:
- The server freezes the backbone weights and distributes only the trainable parameters.
- Edge devices update these parameters on local data.
- The server aggregates the updates and repeats the cycle until convergence.
- Frameworks compared: FedPEAT, FedPepTAO, SLoRA, FedPETuning, FederatedScope-LLM and FATE-LLM. FedDAT and PromptFL (built on CLIP) are discussed for multimodal settings.
- Reported efficiency results:
- SLoRA matches full fine-tuning using sparse updates at about 1% density and cuts training time by around 90%.
- Prefix tuning can work with as little as 0.1% of parameters.
- Edge inference tools cited: llama.cpp, MLC LLM, mllm, llama2.c, InferLLM, TinyChatEngine, NanoLLM and DeepSparse.
Cite
@article{2025,
title = {{面向边缘智能的大模型研究进展}},
author = {王睿 and 张留洋 and 高志涌 and 姜彤雲},
journal = {计算机研究与发展 (Journal of Computer Research and Development)},
year = {2025},
doi = {10.7544/issn1000-1239.202440385},
url = {https://doi.org/10.7544/issn1000-1239.202440385}
}References
- Mobile Computing, 2023, 23(5): 5179−5194 find
- ［37］ Xu Mengwei, Yin Wangsong, Cai Dongqi, et al. A survey of resource-efficient LLM and multimodal foundation models[J]. arXiv preprint, arXiv: 2401.08092, 2024 find
- ［38］ Wan Zhongwei, Wang Xin, Liu Che, et al. Efficient large language models: A survey[J]. arXiv preprint, arXiv: 2312.03863, 2023 find
- ［39］ Miao Xupeng, Oliaro G, Zhang Zhihao, et al. Towards efficient generative large language model serving: A survey from algorithms to systems[J]. arXiv preprint, arXiv: 2312.15234, 2023 find
- ［40］ Kachris C. A survey on hardware accelerators for large language models[J]. arXiv preprint, arXiv: 2401.09890, 2024 find
- ［41］ Zhong Juan, Liu Zheng, Chen Xi. Transformer-based models and hardware acceleration analysis in autonomous driving: A survey[J]. arXiv preprint, arXiv: 2304.10891, 2023 — [arXiv:2304.10891](https://arxiv.org/abs/2304.10891) find
- ［42］ Emani M, Foreman S, Sastry V, et al. A comprehensive performance study of large language models on novel AI accelerators[J]. arXiv preprint, arXiv: 2310.04607, 2023 find
- ［43］ Zhang Xiaodong, Zhang Chaokun, Zhao Jijun. State-of-the-art survey on edge intelligence[J]. Journal of Computer Research and Development, 2023, 60(12): 2749−2769 (in Chinese) find
This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2025-large-models-edge/