🔗 Who cites «Efficient Large Scale Language Modeling with Mixtures of Experts»

99 papers of our library cite it (of the 149,439 whose bibliographies we read) · 28 with a page here, the most cited first · ← back to «Efficient Large Scale Language Modeling with Mixtures of Experts»

1. Emergent Abilities of Large Language ModelsarXiv:2206.07682Jason Wei, Yi Tay, Rishi Bommasani, et al.2022 · 2,269 citations
2. Training Compute-Optimal Large Language ModelsConference on Neural Information Processing Systems (NeurIPS)Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al.2022 · 2,032 citations
3. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelsarXiv:2206.04615 · Transactions on Machine Learning Research, May/2022, https://openreview.net/forum?id=uyTL5BvosjAarohi Srivastava, Abhinav Rastogi, Abhishek Rao, et al.2022 · 1,756 citations
4. OPT: Open Pre-trained Transformer Language ModelsarXiv:2205.01068Susan Zhang, Stephen Roller, Naman Goyal, et al.2022 · 1,755 citations
5. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Conference on Empirical Methods in Natural Language Processing (EMNLP)Sewon Min, Xinxi Lyu, Ari Holtzman, et al.2022 · 1,444 citations
6. GPT-NeoX-20B: An Open-Source Autoregressive Language ModelBigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language ModelsSidney Black, Stella Biderman, Eric Hallahan, et al.2022 · 831 citations
7. GLM-130B: An Open Bilingual Pre-trained ModelarXiv:2210.02414Aohan Zeng, Xiao Liu, Zhengxiao Du, et al.2022 · 786 citations
8. Orca: A Distributed Serving System for Transformer-Based Generative ModelsOSDI 2022Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, et al.2022 · 478 citations
9. The Falcon Series of Open Language ModelsarXiv:2311.16867Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, et al.2023 · 443 citations
10. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeOSDI 2024Amey Agrawal, Nitin Kedia, Ashish Panwar, et al.2024 · 369 citations
11. Large Language Models for Data Annotation and Synthesis: A SurveyConference on Empirical Methods in Natural Language Processing (EMNLP)Zhen Tan, Dawei Li, Song Wang, et al.2024 · 262 citations
12. DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI ScalearXiv:2201.05596Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.2022 · 261 citations
13. OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of GeneralizationarXiv:2212.12017Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, et al.2022 · 244 citations
14. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language ModelsConference on Neural Information Processing Systems (NeurIPS)Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, et al.2022 · 190 citations
15. SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked PrefillsarXiv 2023Amey Agrawal, Ashish Panwar, Jayashree Mohan, et al.2023 · 144 citations
16. Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and FaithfulnessarXiv:2304.11633Bo Li, Gexiang Fang, Yang Yang, et al.2023 · 142 citations
17. Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language ModelsarXiv:2208.03306Margaret Li, Suchin Gururangan, Tim Dettmers, et al.2022 · 114 citations
18. Efficient Large Language Models: A SurveyarXiv:2312.03863Zhongwei Wan, Xin Wang, Che Liu, et al.2023 · 110 citations
19. BLOOM+1: Adding Language Support to BLOOM for Zero-Shot PromptingAnnual Meeting of the Association for Computational Linguistics (ACL)Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, et al.2023 · 102 citations
20. Sparse Upcycling: Training Mixture-of-Experts from Dense CheckpointsarXiv:2212.05055Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al.2022 · 98 citations
21. Accelerating Distributed MoE Training and Inference with LinaarXiv:2210.17223Jiamin Li, Yimin Jiang, Yibo Zhu, et al.2022 · 92 citations
22. Cross-Lingual Consistency of Factual Knowledge in Multilingual Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Jirui Qi, Raquel Fernández, Arianna Bisazza2023 · 90 citations
23. Task Contamination: Language Models May Not Be Few-Shot AnymoreProceedings of the AAAI Conference on Artificial IntelligenceChangmao Li, Jeffrey Flanigan2023 · 89 citations
24. Small Language Models: Survey, Measurements, and InsightsarXiv:2409.15790Zhenyan Lu, Xiang Li, Dongqi Cai, et al.2024 · 88 citations
25. D4: Improving LLM Pretraining via Document De-Duplication and DiversificationConference on Neural Information Processing Systems (NeurIPS)Kushal Tirumala, Daniel Simig, Armen Aghajanyan, et al.2023 · 81 citations
26. Language models scale reliably with over-training and on downstream tasksarXiv:2403.08540Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, et al.2024 · 46 citations
27. PanGu-{\Sigma}: Towards Trillion Parameter Language Model with Sparse Heterogeneous ComputingarXiv:2303.10845Xiaozhe Ren, Pingyi Zhou, Xinfan Meng, et al.2023 · 40 citations
28. Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language ModelsarXiv:2406.06563Tianwen Wei, Bo Zhu, Liang Zhao, et al.2024 · 21 citations

71 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/cited-by/