🔗 Who cites «ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning»

200 papers of our library cite it (of the 149,439 whose bibliographies we read) · 25 with a page here, the most cited first · ← back to «ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning»

1. PaLM: Scaling Language Modeling with PathwaysarXiv:2204.02311Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.2022 · 4,753 citations
2. On the Opportunities and Risks of Foundation ModelsarXiv:2108.07258Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.2021 · 3,160 citations
3. A Survey of Large Language ModelsFrontiers of Computer ScienceWayne Xin Zhao, Kun Zhou, Junyi Li, et al.2023 · 2,875 citations
4. XLS-R: Self-supervised Cross-lingual Speech Representation Learning at ScaleInterspeechArun Babu, Changhan Wang, Andros Tjandra, et al.2021 · 807 citations
5. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language ModelarXiv:2201.11990Shaden Smith, Mostofa Patwary, Brandon Norick, et al.2022 · 615 citations
6. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUarXiv:2303.06865Ying Sheng, Lianmin Zheng, Binhang Yuan, et al.2023 · 402 citations
7. Scaling Speech Technology to 1,000+ LanguagesarXiv:2305.13516Vineel Pratap, Andros Tjandra, Bowen Shi, et al.2023 · 334 citations
8. Challenges and Applications of Large Language ModelsarXiv:2307.10169Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.2023 · 301 citations
9. Reducing Activation Recomputation in Large Transformer ModelsarXiv:2205.05198Vijay Korthikanti, Jared Casper, Sangkug Lym, et al.2022 · 273 citations
10. ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem SolvingarXiv:2309.17452Zhibin Gou, Zhihong Shao, Yeyun Gong, et al.2023 · 225 citations
11. LLM in a flash: Efficient Large Language Model Inference with Limited MemoryAnnual Meeting of the Association for Computational Linguistics (ACL)Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, et al.2024 · 193 citations
12. 8-bit Optimizers via Block-wise QuantizationarXiv:2110.02861Tim Dettmers, Mike Lewis, Sam Shleifer, et al.2021 · 181 citations
13. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer ModelsarXiv:2309.14509Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, et al.2023 · 162 citations
14. Full Parameter Fine-tuning for Large Language Models with Limited ResourcesAnnual Meeting of the Association for Computational Linguistics (ACL)Kai Lv, Yuqing Yang, Tengxiao Liu, et al.2024 · 143 citations
15. Sequence Parallelism: Long Sequence Training from System PerspectiveAnnual Meeting of the Association for Computational Linguistics (ACL)Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, et al.2023 · 129 citations
16. AlphaMath Almost Zero: Process Supervision without ProcessConference on Neural Information Processing Systems (NeurIPS)Guoxin Chen, Minpeng Liao, Chengxi Li, et al.2024 · 120 citations
17. Efficient Large Language Models: A SurveyarXiv:2312.03863Zhongwei Wan, Xin Wang, Che Liu, et al.2023 · 110 citations
18. Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentionUSENIX ATC 2024Bin Gao, Zhuomin He, Puru Sharma, et al.2024 · 105 citations
19. FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel FusionarXiv:2406.06858Li-Wen Chang, Wenlei Bao, Qi Hou, et al.2024 · 67 citations
20. Small LLMs Are Weak Tool Learners: A Multi-LLM AgentConference on Empirical Methods in Natural Language Processing (EMNLP)Weizhou Shen, Chenliang Li, Hongzhan Chen, et al.2024 · 65 citations
21. MARIO: MAth Reasoning with code Interpreter Output - A Reproducible PipelineFindings of the Association for Computational LinguisticsMinpeng Liao, Chengxi Li, Wei Luo, et al.2024 · 64 citations
22. SciAgent: Tool-augmented Language Models for Scientific ReasoningConference on Empirical Methods in Natural Language Processing (EMNLP)Yubo Ma, Zhibin Gou, Junheng Hao, et al.2024 · 55 citations
23. Rationale-Guided Distillation for E-Commerce Relevance Classification: Bridging Large Language Models and Lightweight Cross-EncodersInternational Conference on Computational Linguistics (COLING)Sanjay Agrawal, Faizan Ahemad, Vivek Varadarajan Sembium2025 · 1 citation
24. Joint Optimization of Training Data and Policy in RLHFFindings of the Association for Computational LinguisticsZhuohao Yu, Jiali Zeng, Weizheng Gu, et al.2026
25. Survey on Key Technologies for Large Language Model Pre-training Systems (大语言模型预训练系统关键技术综述)Journal of SoftwareYanjie Gao (高彦杰), Yueguo Chen (陈跃国)2025

175 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rajbhandari-2021-zero-infinity-breaking-gpu-memory/cited-by/