🔗 Who cites «A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity»

98 papers of our library cite it (of the 149,439 whose bibliographies we read) · 27 with a page here, the most cited first · ← back to «A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity»

1. A Survey of Large Language ModelsFrontiers of Computer ScienceWayne Xin Zhao, Kun Zhou, Junyi Li, et al.2023 · 2,875 citations
2. Crosslingual Generalization through Multitask FinetuningarXiv:2211.01786Niklas Muennighoff, Thomas Wang, Lintang Sutawika, et al.2022 · 703 citations
3. Challenges and Applications of Large Language ModelsarXiv:2307.10169Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.2023 · 301 citations
4. RedPajama: an Open Dataset for Training Large Language ModelsConference on Neural Information Processing Systems (NeurIPS)Maurice Weber, Daniel Fu, Quentin Anthony, et al.2024 · 260 citations
5. Aya Model: An Instruction Finetuned Open-Access Multilingual Language ModelarXiv:2402.07827Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, et al.2024 · 253 citations
6. Aya Dataset: An Open-Access Collection for Multilingual Instruction TuningAnnual Meeting of the Association for Computational Linguistics (ACL)Shivalika Singh, Freddie Vargus, Daniel D'souza, et al.2024 · 207 citations
7. DataComp-LM: In search of the next generation of training sets for language modelsConference on Neural Information Processing Systems (NeurIPS)Jeffrey Li, Alex Fang, Georgios Smyrnis, et al.2024 · 191 citations
8. FLASK: Fine-grained Language Model Evaluation based on Alignment Skill SetsarXiv:2307.10928Seonghyeon Ye, Doyoung Kim, Sungdong Kim, et al.2023 · 100 citations
9. A Survey on Data Selection for Language ModelsarXiv:2402.16827Alon Albalak, Yanai Elazar, Sang Michael Xie, et al.2024 · 95 citations
10. What's In My Big Data?arXiv:2310.20707Yanai Elazar, Akshita Bhagia, Ian Magnusson, et al.2023 · 92 citations
11. D4: Improving LLM Pretraining via Document De-Duplication and DiversificationConference on Neural Information Processing Systems (NeurIPS)Kushal Tirumala, Daniel Simig, Armen Aghajanyan, et al.2023 · 81 citations
12. Rho-1: Not All Tokens Are What You NeedarXiv:2404.07965Zhenghao Lin, Zhibin Gou, Yeyun Gong, et al.2024 · 59 citations
13. Stable LM 2 1.6B Technical ReportarXiv:2402.17834Marco Bellagente, Jonathan Tow, Dakota Mahan, et al.2024 · 46 citations
14. CulturalBench: A Robust, Diverse and Challenging Benchmark for Measuring LMs' Cultural Knowledge Through Human-AI Red-TeamingAnnual Meeting of the Association for Computational Linguistics (ACL)Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, et al.2025 · 38 citations
15. On the Diversity of Synthetic Data and its Impact on Training Large Language ModelsarXiv:2410.15226Hao Chen, Abdul Waheed, Xiang Li, et al.2024 · 35 citations
16. What is the Role of Small Models in the LLM Era: A SurveyComputational LinguisticsLihu Chen, Gael Varoquaux2024 · 23 citations
17. AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data FiltersAnnual Meeting of the Association for Computational Linguistics (ACL)Li Lucy, Suchin Gururangan, Luca Soldaini, et al.2024 · 15 citations
18. Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase PretrainingarXiv:2412.15285Steven Feng, Shrimai Prabhumoye, Kezhi Kong, et al.2024 · 12 citations
19. TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning CompressionarXiv:2506.02678Zhong-Zhi Li, Xiao Liang, Zihao Tang, et al.2025 · 8 citations
20. Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum LearningConference of the European Chapter of the Association for Computational Linguistics (EACL)Yang Zhang, Amr Mohamed, Hadi Abdine, et al.2026 · 7 citations
21. Improving Romanian LLM Pretraining Data using Diversity and Quality FilteringWorkshop on Language Models for Low-Resource LanguagesVlad-Andrei Negoiță, Mihai Masala, Traian Rebedea2026 · 2 citations
22. The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language ModelsarXiv:2510.13996Lukas Gienapp, Christopher Schroder, Stefan Schweter, et al.2025 · 2 citations
23. Aligned Probing: Relating Toxic Behavior and Model InternalsTransactions of the Association for Computational Linguistics (TACL)Andreas Waldis, Vagrant Gautam, Anne Lauscher, et al.2026 · 1 citation
24. Fiction Flows: A Replication and Reinterpretation of Narrative SequentialityAnnual Meeting of the Association for Computational Linguistics (ACL)Andrew Piper, Sil Hamilton, Haiqi Zhou, et al.2026
25. How Does "English (US)" Become the Default? Triangulating Structural Bias Towards American English Across the LLM PipelinearXiv:2604.04204Mir Tafseer Nayeem, Davood Rafiei2026
26. Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model BehaviorTransactions of the Association for Computational Linguistics (TACL)Rahul Nadkarni, Yanai Elazar, Hila Gonen, et al.2026
27. The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose ErrorsWorkshop on Innovative Use of NLP for Building Educational Applications (BEA)Li Lucy, Albert Zhang, Nathan Anderson, et al.2026

71 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/longpre-2023-pretrainer-s-guide-training-data/cited-by/