🔗 Who cites «BASE Layers: Simplifying Training of Large, Sparse Models»

116 papers of our library cite it (of the 149,439 whose bibliographies we read) · 27 with a page here, the most cited first · ← back to «BASE Layers: Simplifying Training of Large, Sparse Models»

1. The Llama 3 Herd of ModelsarXiv:2407.21783Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al.2024 · 7,682 citations
2. Efficient Transformers: A SurveyACM Computing SurveysYi Tay, Mostafa Dehghani, Dara Bahri, et al.2020 · 1,539 citations
3. Challenges and Applications of Large Language ModelsarXiv:2307.10169Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.2023 · 301 citations
4. BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagearXiv:2208.03188Kurt Shuster, Jing Xu, Mojtaba Komeili, et al.2022 · 241 citations
5. 8-bit Optimizers via Block-wise QuantizationarXiv:2110.02861Tim Dettmers, Mike Lewis, Sam Shleifer, et al.2021 · 181 citations
6. Tutel: Adaptive Mixture-of-Experts at ScaleMLSys 2023Changho Hwang, Wei Cui, Yifan Xiong, et al.2022 · 168 citations
7. AdaMix: Mixture-of-Adaptations for Parameter-efficient Model TuningConference on Empirical Methods in Natural Language Processing (EMNLP)Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, et al.2022 · 135 citations
8. Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt CompletionFindings of the Association for Computational LinguisticsKurt Shuster, Mojtaba Komeili, Leonard Adolphs, et al.2022 · 132 citations
9. Scaling Vision with Sparse Mixture of ExpertsarXiv:2106.05974Carlos Riquelme, Joan Puigcerver, Basil Mustafa, et al.2021 · 128 citations
10. MoEfication: Transformer Feed-forward Layers are Mixtures of ExpertsFindings of the Association for Computational LinguisticsZhengyan Zhang, Yankai Lin, Zhiyuan Liu, et al.2022 · 115 citations
11. Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language ModelsarXiv:2208.03306Margaret Li, Suchin Gururangan, Tim Dettmers, et al.2022 · 114 citations
12. A Survey on Efficient Inference for Large Language ModelsarXiv:2404.14294Zixuan Zhou, Xuefei Ning, Ke Hong, et al.2024 · 113 citations
13. Efficient Large Scale Language Modeling with Mixtures of ExpertsarXiv:2112.10684Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al.2021 · 113 citations
14. Efficient Large Language Models: A SurveyarXiv:2312.03863Zhongwei Wan, Xin Wang, Che Liu, et al.2023 · 110 citations
15. CPM-2: Large-scale Cost-effective Pre-trained Language ModelsAI OpenZhengyan Zhang, Yuxian Gu, Xu Han, et al.2021 · 102 citations
16. Sparse Upcycling: Training Mixture-of-Experts from Dense CheckpointsarXiv:2212.05055Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al.2022 · 98 citations
17. Accelerating Distributed MoE Training and Inference with LinaarXiv:2210.17223Jiamin Li, Yimin Jiang, Yibo Zhu, et al.2022 · 92 citations
18. LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-TrainingConference on Empirical Methods in Natural Language Processing (EMNLP)Tong Zhu, Xiaoye Qu, Daize Dong, et al.2024 · 85 citations
19. Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language ModelsarXiv:2305.14705Sheng Shen, Le Hou, Yanqi Zhou, et al.2023 · 70 citations
20. Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction TuningarXiv:2309.05444Ted Zadouri, Ahmet Ustun, Arash Ahmadian, et al.2023 · 69 citations
21. OpenMoE: An Early Effort on Open Mixture-of-Experts Language ModelsarXiv:2402.01739Fuzhao Xue, Zian Zheng, Yao Fu, et al.2024 · 64 citations
22. ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationarXiv:2112.12731Shuohuan Wang, Yu Sun, Yang Xiang, et al.2021 · 63 citations
23. Kimi K3: Open Frontier IntelligencearXiv:2607.24653Kimi Team, Tongtong Bai, Yifan Bai, et al.2026 · 40 citations
24. Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language ModelsarXiv:2305.09955Shangbin Feng, Weijia Shi, Yuyang Bai, et al.2023 · 35 citations
25. ConPET: Continual Parameter-Efficient Tuning for Large Language ModelsarXiv:2309.14763Chenyang Song, Xu Han, Zheni Zeng, et al.2023 · 14 citations
26. PanGu-π: Enhancing Language Model Architectures via Nonlinearity CompensationarXiv:2312.17276Yunhe Wang, Hanting Chen, Yehui Tang, et al.2023 · 10 citations
27. GRIN: GRadient-INformed MoEarXiv:2409.12136Liyuan Liu, Young Jin Kim, Shuohang Wang, et al.2024 · 7 citations

89 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/cited-by/