Elamur Research⌕
☰
home / papers / Scaling Laws for Neural Language Models / cited by / page 2

Who cites «Scaling Laws for Neural Language Models»

1,823 papers of our library cite it (of the 141,840 whose bibliographies we read) · 203 with a page here, the most cited first · page 2 of 3 · 101–200 · ← back to «Scaling Laws for Neural Language Models»

1 2 3

  1. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?Ronen Eldan, Yuanzhi Li2023 · 202 citations
  2. COCO-LM: Correcting and Contrasting Text Sequences for Language Model PretrainingarXiv:2102.08473Yu Meng, Chenyan Xiong, Payal Bajaj, et al.2021 · 196 citations
  3. RRHF: Rank Responses to Align Language Models with Human Feedback without tearsarXiv:2304.05302Zheng Yuan, Hongyi Yuan, Chuanqi Tan, et al.2023 · 194 citations
  4. Merging Models with Fisher-Weighted AveragingConference on Neural Information Processing Systems (NeurIPS)Michael Matena, Colin Raffel2021 · 191 citations
  5. LawBench: Benchmarking Legal Knowledge of Large Language ModelsarXiv:2309.16289Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, et al.2023 · 186 citations
  6. OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text DocumentsConference on Neural Information Processing Systems (NeurIPS)Hugo Laurençon, Lucile Saulnier, Leo Tronchon, et al.2023 · 183 citations
  7. Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language ModelsConference on Neural Information Processing Systems (NeurIPS)Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, et al.2022 · 183 citations
  8. Pre-training via ParaphrasingarXiv:2006.15020Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, et al.2020 · 181 citations
  9. Knowledge Conflicts for LLMs: A SurveyConference on Empirical Methods in Natural Language Processing (EMNLP)Rongwu Xu, Zehan Qi, Zhijiang Guo, et al.2024 · 179 citations
  10. Effective Long-Context Scaling of Foundation ModelsarXiv:2309.16039Wenhan Xiong, Jingyu Liu, Igor Molybog, et al.2023 · 177 citations
  11. 8-bit Optimizers via Block-wise QuantizationarXiv:2110.02861Tim Dettmers, Mike Lewis, Sam Shleifer, et al.2021 · 177 citations
  12. ShortGPT: Layers in Large Language Models are More Redundant Than You ExpectarXiv:2403.03853Xin Men, Mingyu Xu, Qingyu Zhang, et al.2024 · 167 citations
  13. Large Language Diffusion ModelsConference on Neural Information Processing Systems (NeurIPS)Shen Nie, Fengqi Zhu, Zebin You, et al.2025 · 164 citations
  14. BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex InstructionsarXiv:2406.15877Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, et al.2024 · 163 citations
  15. Scaling Synthetic Data Creation with 1,000,000,000 PersonasarXiv:2406.20094Tao Ge, Xin Chan, Xiaoyang Wang, et al.2024 · 160 citations
  16. Leveraging Large Language Models for Multiple Choice Question AnsweringarXiv:2210.12353Joshua Robinson, Christopher Michael Rytting, David Wingate2022 · 157 citations
  17. DataComp-LM: In search of the next generation of training sets for language modelsConference on Neural Information Processing Systems (NeurIPS)Jeffrey Li, Alex Fang, Georgios Smyrnis, et al.2024 · 156 citations
  18. HunyuanVideo: A Systematic Framework For Large Video Generative ModelsarXiv:2412.03603Weijie Kong, Qi Tian, Zijian Zhang, et al.2024 · 155 citations
  19. Language Model Behavior: A Comprehensive SurveyComputational LinguisticsTyler A. Chang, Benjamin K. Bergen2024 · 148 citations
  20. The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual DatasetarXiv:2303.03915Hugo Laurençon, Lucile Saulnier, Thomas Wang, et al.2023 · 147 citations
  21. How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data CompositionAnnual Meeting of the Association for Computational Linguistics (ACL)Guanting Dong, Hongyi Yuan, Keming Lu, et al.2024 · 145 citations
  22. Efficient Methods for Natural Language Processing: A SurveyTransactions of the Association for Computational Linguistics (TACL)Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.2023 · 145 citations
  23. PMC-LLaMA: Towards Building Open-source Language Models for MedicinearXiv:2304.14454Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, et al.2023 · 144 citations
  24. MiniLLM: On-Policy Distillation of Large Language ModelsarXiv:2306.08543Yuxian Gu, Li Dong, Furu Wei, et al.2023 · 143 citations
  25. Rethinking embedding coupling in pre-trained language modelsarXiv:2010.12821Hyung Won Chung, Thibault Févry, Henry Tsai, et al.2020 · 142 citations
  26. DeepInception: Hypnotize Large Language Model to Be JailbreakerarXiv:2311.03191Xuan Li, Zhanke Zhou, Jianing Zhu, et al.2023 · 138 citations
  27. Efficient Training of Language Models to Fill in the MiddlearXiv:2207.14255Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, et al.2022 · 138 citations
  28. Many-Shot In-Context LearningConference on Neural Information Processing Systems (NeurIPS)Rishabh Agarwal, Avi Singh, Lei M. Zhang, et al.2024 · 134 citations
  29. Few-Shot Self-Rationalization with Natural Language PromptsFindings of the Association for Computational LinguisticsAna Marasovic, Iz Beltagy, Doug Downey, et al.2022 · 134 citations
  30. Formal Mathematics Statement Curriculum LearningarXiv:2202.01344Stanislas Polu, Jesse Michael Han, Kunhao Zheng, et al.2022 · 132 citations
  31. EuroLLM: Multilingual Language Models for EuropearXiv:2409.16235Pedro Henrique Martins, Patrick Fernandes, Joao Alves, et al.2024 · 131 citations
  32. A Survey on Recent Advances in LLM-Based Multi-turn Dialogue SystemsACM Computing SurveysZihao Yi, Jiarui Ouyang, Zhe Xu, et al.2024 · 129 citations
  33. Scaling Sentence Embeddings with Large Language ModelsFindings of the Association for Computational LinguisticsTing Jiang, Shaohan Huang, Zhongzhi Luan, et al.2024 · 126 citations
  34. Large Language Model Alignment: A SurveyarXiv:2309.15025Tianhao Shen, Renren Jin, Yufei Huang, et al.2023 · 123 citations
  35. INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic FeedbackConference on Empirical Methods in Natural Language Processing (EMNLP)Wenda Xu, Danqing Wang, Liangming Pan, et al.2023 · 122 citations
  36. Knowledge of cultural moral norms in large language modelsAnnual Meeting of the Association for Computational Linguistics (ACL)Aida Ramezani, Yang Xu2023 · 119 citations
  37. The case for 4-bit precision: k-bit Inference Scaling LawsarXiv:2212.09720Tim Dettmers, Luke Zettlemoyer2022 · 119 citations
  38. How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVENTransactions of the Association for Computational Linguistics (TACL)R. Thomas McCoy, Paul Smolensky, Tal Linzen, et al.2023 · 116 citations
  39. The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-TuningConference on Empirical Methods in Natural Language Processing (EMNLP)Seungone Kim, Se Joo, Doyoung Kim, et al.2023 · 116 citations
  40. Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language ModelsarXiv:2408.00724Yangzhen Wu, Zhiqing Sun, Shanda Li, et al.2024 · 114 citations
  41. ALLaM: Large Language Models for Arabic and EnglisharXiv:2407.15390M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, et al.2024 · 113 citations
  42. Neural Networks and the Chomsky HierarchyarXiv:2207.02098Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, et al.2022 · 110 citations
  43. Data Engineering for Scaling Language Models to 128K ContextarXiv:2402.10171Yao Fu, Rameswar Panda, Xinyao Niu, et al.2024 · 107 citations
  44. Scaling Laws for TransferarXiv:2102.01293Danny Hernandez, Jared Kaplan, Tom Henighan, et al.2021 · 102 citations
  45. Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation InstructionsTransactions of the Association for Computational Linguistics (TACL)Jiahuan Li, Hao Zhou, Shujian Huang, et al.2024 · 99 citations
  46. Inverse Scaling: When Bigger Isn't BetterarXiv:2306.09479 · Transactions on Machine Learning Research (TMLR), 10/2023, https://openreview.net/forum?id=DwgRm72GQFIan R. McKenzie, Alexander Lyzhov, Michael Pieler, et al.2023 · 99 citations
  47. Inseq: An Interpretability Toolkit for Sequence Generation ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Gabriele Sarti, Nils Feldhus, Ludwig Sickert, et al.2023 · 96 citations
  48. SemDeDup: Data-efficient learning at web-scale through semantic deduplicationarXiv:2303.09540Amro Abbas, Kushal Tirumala, Dániel Simig, et al.2023 · 94 citations
  49. Exploring the Potential of ChatGPT on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse RelationsFindings of the Association for Computational LinguisticsChunkit Chan, Cheng Jiayang, Weiqi Wang, et al.2024 · 92 citations
  50. Skywork: A More Open Bilingual Foundation ModelarXiv:2310.19341Tianwen Wei, Liang Zhao, Lichang Zhang, et al.2023 · 91 citations
  51. Language Modeling Is CompressionarXiv:2309.10668Gregoire Deletang, Anian Ruoss, Paul-Ambroise Duquenne, et al.2023 · 89 citations
  52. Small Language Models: Survey, Measurements, and InsightsarXiv:2409.15790Zhenyan Lu, Xiang Li, Dongqi Cai, et al.2024 · 88 citations
  53. A Survey of Large Language Models in Medicine: Progress, Application, and ChallengearXiv:2311.05112Hongjian Zhou, Fenglin Liu, Boyang Gu, et al.2023 · 87 citations
  54. PromptNER: Prompting For Named Entity RecognitionarXiv:2305.15444Dhananjay Ashok, Zachary C. Lipton2023 · 86 citations
  55. A Survey on Data Selection for Language ModelsarXiv:2402.16827Alon Albalak, Yanai Elazar, Sang Michael Xie, et al.2024 · 85 citations
  56. When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource LanguagesConference on Empirical Methods in Natural Language Processing (EMNLP)Tyler A. Chang, Catherine Arnett, Zhuowen Tu, et al.2024 · 85 citations
  57. Multilingual Large Language Models Are Not (Yet) Code-SwitchersConference on Empirical Methods in Natural Language Processing (EMNLP)Ruochen Zhang, Samuel Cahyawijaya, Jan Christian Blaise Cruz, et al.2023 · 85 citations
  58. CoLT5: Faster Long-Range Transformers with Conditional ComputationConference on Empirical Methods in Natural Language Processing (EMNLP)Joshua Ainslie, Tao Lei, Michiel de Jong, et al.2023 · 83 citations
  59. D4: Improving LLM Pretraining via Document De-Duplication and DiversificationConference on Neural Information Processing Systems (NeurIPS)Kushal Tirumala, Daniel Simig, Armen Aghajanyan, et al.2023 · 79 citations
  60. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?Findings of the Association for Computational LinguisticsYi Tay, Mostafa Dehghani, Samira Abnar, et al.2023 · 79 citations
  61. Coreference Resolution through a seq2seq Transition-Based SystemTransactions of the Association for Computational Linguistics (TACL)Bernd Bohnet, Chris Alberti, Michael Collins2023 · 77 citations
  62. SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-ScalingarXiv:2312.15166Dahyun Kim, Chanjun Park, Sanghoon Kim, et al.2023 · 75 citations
  63. ReLoRA: High-Rank Training Through Low-Rank UpdatesarXiv:2307.05695Vladislav Lialin, Namrata Shivagunde, Sherin Muckatira, et al.2023 · 74 citations
  64. Iterative Translation Refinement with Large Language ModelsAnnual Conference of the European Association for Machine Translation (EAMT)Pinzhen Chen, Zhicheng Guo, Barry Haddow, et al.2024 · 73 citations
  65. OR-Bench: An Over-Refusal Benchmark for Large Language ModelsarXiv:2405.20947Justin Cui, Wei-Lin Chiang, Ion Stoica, et al.2024 · 73 citations
  66. KMMLU: Measuring Massive Multitask Language Understanding in KoreanConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Guijin Son, Hanwool Lee, Sungdong Kim, et al.2025 · 72 citations
  67. It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense ReasoningFindings of the Association for Computational LinguisticsAlexey Tikhonov, Max Ryabinin2021 · 69 citations
  68. Contrastive Decoding Improves Reasoning in Large Language ModelsarXiv:2309.09117Sean O'Brien, Mike Lewis2023 · 67 citations
  69. Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction TuningarXiv:2309.05444Ted Zadouri, Ahmet Ustun, Arash Ahmadian, et al.2023 · 66 citations
  70. Seed1.5-VL Technical ReportarXiv:2505.07062Dong Guo, Faming Wu, Feida Zhu, et al.2025 · 64 citations
  71. SpiRit-LM: Interleaved Spoken and Written Language ModelTransactions of the Association for Computational Linguistics (TACL)Tu Anh Nguyen, Benjamin Muller, Bokai Yu, et al.2025 · 61 citations
  72. TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic VarietiesArabic Natural Language Processing Conference (ArabicNLP)Karima Kadaoui, Samar M. Magdy, Abdul Waheed, et al.2023 · 60 citations
  73. Scaling Laws of RoPE-based ExtrapolationarXiv:2310.05209Xiaoran Liu, Hang Yan, Shuo Zhang, et al.2023 · 53 citations
  74. Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling LawsarXiv:2401.00448Nikhil Sardana, Jacob Portes, Sasha Doubov, et al.2024 · 51 citations
  75. Scaling Laws with Vocabulary: Larger Models Deserve Larger VocabulariesarXiv:2407.13623Chaofan Tao, Qian Liu, Longxu Dou, et al.2024 · 42 citations
  76. Transcending Scaling Laws with 0.1% Extra ComputearXiv:2210.11399Yi Tay, Jason Wei, Hyung Won Chung, et al.2022 · 41 citations
  77. Scaling Laws for Generative Mixed-Modal Language ModelsarXiv:2301.03728Armen Aghajanyan, Lili Yu, Alexis Conneau, et al.2023 · 40 citations
  78. Observational Scaling Laws and the Predictability of Language Model PerformancearXiv:2405.10938Yangjun Ruan, Chris J. Maddison, Tatsunori Hashimoto2024 · 30 citations
  79. Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling ApproachKoren Lazar, Benny Saret, Asaf Yehudai, et al.2021 · 30 citations
  80. Scaling Laws and Compute-Optimal Training Beyond Fixed Training DurationsConference on Neural Information Processing Systems (NeurIPS)Alexander Hägele, Elie Bakouch, Atli Kosson, et al.2024 · 26 citations
  81. Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal SamplingarXiv:2408.16737Hritik Bansal, Arian Hosseini, Rishabh Agarwal, et al.2024 · 26 citations
  82. Distillation Scaling LawsarXiv:2502.08606Dan Busbridge, Amitis Shidani, Floris Weers, et al.2025 · 23 citations
  83. A Large and Diverse Arabic Corpus for Language ModelingProcedia Computer ScienceAbbas Raza Ali, Muhammad Ajmal Siddiqui, Rema Algunaibet, et al.2022 · 16 citations
  84. A Simple and Strong Baseline for End-to-End Neural RST-style Discourse ParsingFindings of the Association for Computational LinguisticsNaoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, et al.2022 · 16 citations
  85. Resolving Discrepancies in Compute-Optimal Scaling of Language ModelsConference on Neural Information Processing Systems (NeurIPS)Tomer Porian, Mitchell Wortsman, Jenia Jitsev, et al.2024 · 14 citations
  86. Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models — The Story Goes OnarXiv:2407.08348Liang Zeng, Liangjun Zhong, Liang Zhao, et al.2024 · 14 citations
  87. TigerCoder: A Novel Suite of LLMs for Code Generation in BanglaLanguage Resources and Evaluation Conference (LREC)Nishat Raihan, Antonios Anastasopoulos, Marcos Zampieri2026 · 13 citations
  88. When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM ReasoningarXiv:2504.01005Nishad Singhi, Hritik Bansal, Arian Hosseini, et al.2025 · 12 citations
  89. Multilingual Sequence-to-Sequence Models for Hebrew NLPMatan Eyal, Hila Noga, Roee Aharoni, et al.2023 · 9 citations
  90. Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum LearningConference of the European Chapter of the Association for Computational Linguistics (EACL)Yang Zhang, Amr Mohamed, Hadi Abdine, et al.2026 · 7 citations
  91. Kuwain 1.5B: An Arabic SLM via Language InjectionarXiv:2504.15120Khalil Hennara, Sara Chrouf, Mohamed Motaism Hamed, et al.2025 · 4 citations
  92. Scaling Laws for Code: A More Data-Hungry RegimeAnnual Meeting of the Association for Computational Linguistics (ACL)Xianzhen Luo, Wenzhen Zheng, Qingfu Zhu, et al.2026 · 3 citations
  93. TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in HebrewAnita Gelboim, Elior Sulem2025 · 3 citations
  94. Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language ModelsTransactions of the Association for Computational Linguistics (TACL)Abishek Thamma, Micha Heilbron2026 · 1 citation
  95. Batch-wise Convergent Pre-training: Step-by-Step Learning Inspired by Child Language DevelopmentBabyLM WorkshopKo Yoshida, Daiki Shiono, Kai Sato, et al.2025 · 1 citation
  96. A Survey on Parameter-efficient Fine-tuning of Large Models: Techniques, Trends, and Challenges (大模型参数高效微调方法综述: 技术、趋势与挑战)Acta Automatica Sinica (自动化学报)Anda Tang (唐岸达), Zhouchen Lin (林宙辰)2026
  97. Detecting Latin in Historical Books with Large Language Models: A Multimodal BenchmarkConference of the European Chapter of the Association for Computational Linguistics (EACL)Yu Wu, Ke Shu, Jonas Fischer, et al.2026
  98. Investigating Memorization in Language Models Trained via Knowledge DistillationLanguage Resources and Evaluation Conference (LREC)Maarten Mäcking, Michaela Regneri2026
  99. MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel DetectionDavid M. Smiley2026
  100. Quality and Appropriateness of Large Text Datasets for Irish NLPSIGUL Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered LanguagesAbigail Walsh, Mark Andrade, Jane Lauren Adkins, et al.2026

1 2 3

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/cited-by/page/2/