Elamur Research⌕
☰
home / authors / Yuanzhi Li / all works

Yuanzhi Li — all works

125 works · page 1 of 2 · 1–100 · ← Yuanzhi Li

1 2

  1. LoRA: Low-Rank Adaptation of Large Language ModelsarXiv:2106.09685Edward J. Hu, Yelong Shen, Phillip Wallis, et al.2021 · 8,661 citations
  2. Sparks of Artificial General Intelligence: Early experiments with GPT-4arXiv:2303.12712Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, et al.2023 · 2,439 citations
  3. A Convergence Theory for Deep Learning via Over-Parameterization ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li, Zhao Song2018 · 696 citations
  4. Textbooks Are All You Need ↗arXiv:2306.11644 · no summary yetSuriya Gunasekar et al.2023 · 490 citations
  5. A Latent Variable Model Approach to PMI-based Word EmbeddingsTransactions of the Association for Computational Linguistics (TACL)Sanjeev Arora, Yuanzhi Li, Yingyu Liang, et al.2016 · 372 citations
  6. Textbooks Are All You Need II: phi-1.5 technical reportarXiv:2309.05463Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, et al.2023 · 336 citations
  7. A Theoretical Analysis of NDCG Type Ranking Measures ↗arXiv (Cornell University) · no summary yetYining Wang et al.2013 · 336 citations
  8. Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine ↗arXiv:2311.16452 · no summary yetHarsha Nori et al.2023 · 296 citations
  9. Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yingyu Liang2018 · 295 citations
  10. Convergence Analysis of Two-layer Neural Networks with ReLU Activation ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yuan Yang2017 · 293 citations
  11. Linear Algebraic Structure of Word Senses, with Applications to PolysemyTransactions of the Association for Computational Linguistics (TACL)Sanjeev Arora, Yuanzhi Li, Yingyu Liang, et al.2018 · 254 citations
  12. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?Ronen Eldan, Yuanzhi LiEnglish · 213 citations
  13. Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li, Yingyu Liang2018 · 193 citations
  14. Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2020 · 188 citations
  15. Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Colin Wei, Tengyu Ma2019 · 138 citations
  16. Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees ↗arXiv (Cornell University) · no summary yetYuping Luo et al.2018 · 125 citations
  17. On the Convergence Rate of Training Recurrent Neural Networks ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li, Zhao Song2018 · 120 citations
  18. Physics of Language Models: Part 3.1, Knowledge Storage and ExtractionarXiv:2309.14316Zeyuan Allen-Zhu, Yuanzhi Li2023 · 116 citations
  19. What Can ResNet Learn Efficiently, Going Beyond Kernels? ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2019 · 116 citations
  20. Feature Purification: How Adversarial Training Performs Robust Deep Learning ↗Annual Symposium on Foundations of Computer Science · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2020 · 105 citations
  21. O1 Replication Journey: A Strategic Progress Report — Part 1arXiv:2410.18982Yiwei Qin, Xuefeng Li, Haoyang Zou, et al.2024 · 86 citations
  22. Physics of Language Models: Part 3.2, Knowledge ManipulationarXiv:2309.14402Zeyuan Allen-Zhu, Yuanzhi Li2023 · 72 citations
  23. Sparsity, variance and curvature in multi-armed bandits ↗arXiv (Cornell University) · no summary yetSébastien Bubeck, Michael B. Cohen, Yuanzhi Li2017 · 69 citations
  24. Much Faster Algorithms for Matrix Scaling ↗Annual Symposium on Foundations of Computer Science · no summary yetZeyuan Allen-Zhu et al.2017 · 68 citations
  25. Neon2: Finding Local Minima via First-Order Oracles ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2017 · 63 citations
  26. First Efficient Convergence for Streaming k-PCA: A Global, Gap-Free, and Near-Optimal Rate ↗Annual Symposium on Foundations of Computer Science · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2017 · 62 citations
  27. Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning ProcessarXiv:2407.20311Tian Ye, Zicheng Xu, Yuanzhi Li, et al.2024 · 58 citations
  28. Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws ↗SSRN Electronic Journal · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2025 · 53 citations
  29. Can SGD Learn Recurrent Neural Networks with Provable Generalization? ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2019 · 51 citations
  30. Towards Understanding the Mixture-Of-Experts Layer in Deep Learning ↗neural information processing systems · no summary yetZixiang Chen et al.2022 · 49 citations
  31. Near-optimal discrete optimization for experimental design: a regret minimization approach ↗Mathematical Programming · no summary yetZeyuan Allen-Zhu et al.2017 · 48 citations
  32. An Alternative View: When Does SGD Escape Local Minima? ↗arXiv (Cornell University) · no summary yetRobert Kleinberg, Yuanzhi Li, Yuan Yang2018 · 46 citations
  33. Operator scaling via geodesically convex optimization, invariant theory and polynomial identity testing ↗ACM SIGACT Symposium on Theory of Computing (STOC) · no summary yetZeyuan Allen-Zhu et al.2018 · 46 citations
  34. TinyGSM: achieving >80% on GSM8k with small language models ↗arXiv:2312.09241 · no summary yetBingbin Liu et al.2023 · 45 citations
  35. Physics of Language Models: Part 1, Learning Hierarchical Language Structures ↗arXiv:2305.13673 · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2023 · 42 citations
  36. SmartPlay: A Benchmark for LLMs as Intelligent Agents ↗arXiv (Cornell University) · no summary yetYue Wu et al.2023 · 41 citations
  37. PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections ↗OSDI 2021 · no summary yetHaojie Wang et al.2021 · 40 citations
  38. Chasing Nested Convex Bodies Nearly Optimally ↗Society for Industrial and Applied Mathematics eBooks · no summary yetSébastien Bubeck et al.2019 · 40 citations
  39. LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2016 · 40 citations
  40. Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Tengyu Ma, Hongyang Zhang2017 · 38 citations
  41. Towards Understanding Mixture of Experts in Deep Learning ↗arXiv (Cornell University) · no summary yetZixiang Chen et al.2022 · 37 citations
  42. Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability ↗arXiv (Cornell University) · no summary yetCohen, Jeremy M. et al.2021 · 36 citations
  43. VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding? ↗arXiv (Cornell University) · no summary yetJunpeng Liu et al.2024 · 33 citations
  44. Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Tengyu Ma, Hongyang Zhang2020 · 33 citations
  45. Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu et al.2017 · 32 citations
  46. Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems ↗arXiv:2408.16293 · no summary yetTian Ye et al.2024 · 31 citations
  47. Positional Description Matters for Transformers Arithmetic ↗arXiv:2311.14737 · no summary yetRuoqi Shen et al.2023 · 30 citations
  48. An homotopy method for l p regression provably beyond self-concordance and in input-sparsity time ↗ACM SIGACT Symposium on Theory of Computing (STOC) · no summary yetSébastien Bubeck et al.2018 · 30 citations
  49. Length Generalization in Arithmetic Transformers ↗arXiv:2306.15400 · no summary yetSamy Jelassi et al.2023 · 29 citations
  50. Near-optimal method for highly smooth convex optimization ↗arXiv (Cornell University) · no summary yetSébastien Bubeck et al.2018 · 28 citations
  51. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions ↗arXiv (Cornell University) · no summary yetSitan Chen et al.2022 · 25 citations
  52. Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yingyu Liang, Andrej Risteski2016 · 25 citations
  53. Backward Feature Correction: How Deep Learning Performs Deep Learning ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2020 · 24 citations
  54. Vision Transformers Provably Learn Spatial Structure ↗neural information processing systems · no summary yetSamy Jelassi, Michael Sander, Yuanzhi Li2022 · 23 citations
  55. Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2017 · 23 citations
  56. Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning ↗arXiv (Cornell University) · no summary yetZixin Wen, Yuanzhi Li2021 · 22 citations
  57. Complexity of Highly Parallel Non-Smooth Convex Optimization ↗arXiv (Cornell University) · no summary yetSébastien Bubeck et al.2019 · 22 citations
  58. A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model ↗Society for Industrial and Applied Mathematics eBooks · no summary yetXi Chen, Yuanzhi Li, Jieming Mao2018 · 22 citations
  59. Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlations ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yingyu Liang2017 · 22 citations
  60. An optimal algorithm for bandit convex optimization ↗arXiv (Cornell University) · no summary yetElad Hazan, Yuanzhi Li2016 · 21 citations
  61. RAND-WALK: A Latent Variable Model Approach to Word Embeddings ↗arXiv (Cornell University) · no summary yetSanjeev Arora et al.2015 · 21 citations
  62. Efficient RLHF: Reducing the Memory Usage of PPO ↗arXiv:2309.00754 · no summary yetMichael Santacroce et al.2023 · 20 citations
  63. What Matters In The Structured Pruning of Generative Language Models? ↗arXiv (Cornell University) · no summary yetMichael Santacroce et al.2023 · 20 citations
  64. Plan, Eliminate, and Track — Language Models are Good Teachers for Embodied Agents ↗arXiv (Cornell University) · no summary yetYue Wu et al.2023 · 20 citations
  65. Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals ↗neural information processing systems · no summary yetYue Wu et al.2023 · 20 citations
  66. Near Optimal Methods for Minimizing Convex Functions with Lipschitz p-th Derivatives ↗Conference on Learning Theory · no summary yetAlexander Vladimirovich Gasnikov et al.2019 · 20 citations
  67. Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2016 · 20 citations
  68. The probability flow ODE is provably fast ↗neural information processing systems · no summary yetSitan Chen et al.2023 · 19 citations
  69. SPRING: Studying the Paper and Reasoning to Play Games ↗arXiv (Cornell University) · no summary yetYue Wu et al.2023 · 19 citations
  70. Recovery guarantee of weighted low-rank approximation via alternating minimization ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yingyu Liang, Andrej Risteski2016 · 17 citations
  71. LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks ↗Proceedings of the 31st International Conference on Computational Linguistics: Industry Track · no summary yetAkshara Prabhakar et al.2025 · 14 citations
  72. How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding ↗arXiv (Cornell University) · no summary yetYuchen Li, Yuanzhi Li, Andrej Risteski2023 · 14 citations
  73. Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization ↗arXiv (Cornell University) · no summary yetDifan Zou et al.2021 · 13 citations
  74. Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Sébastien Bubeck, Yuanzhi Li2018 · 13 citations
  75. Toward Understanding Why Adam Converges Faster Than SGD for Transformers ↗arXiv (Cornell University) · no summary yetYan Pan, Yuanzhi Li2023 · 12 citations
  76. Online Improper Learning with an Approximation Oracle ↗arXiv (Cornell University) · no summary yetElad Hazan et al.2018 · 12 citations
  77. Towards understanding how momentum improves generalization in deep learning ↗arXiv (Cornell University) · no summary yetSamy Jelassi, Yuanzhi Li2022 · 11 citations
  78. Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2021 · 11 citations
  79. When can Wasserstein GANs minimize Wasserstein Distance ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Zehao Dou2020 · 11 citations
  80. Learning Mixtures of Linear Regressions with Nearly Optimal Complexity ↗arXiv (Cornell University) · no summary yetYuanzhi Li, Yingyu Liang2018 · 10 citations
  81. A law of robustness for two-layers neural networks ↗arXiv (Cornell University) · no summary yetSébastien Bubeck, Yuanzhi Li, Anant Raj2020 · 9 citations
  82. Algorithms and matching lower bounds for approximately-convex optimization ↗neural information processing systems · no summary yetYuanzhi Li, Andrej Risteski2016 · 9 citations
  83. When Is Generalizable Reinforcement Learning Tractable? ↗arXiv (Cornell University) · no summary yetDhruv Malik, Yuanzhi Li, Pradeep Ravikumar2021 · 8 citations
  84. Faster Principal Component Regression and Stable Matrix Chebyshev Approximation ↗arXiv (Cornell University) · no summary yetZeyuan Allen-Zhu, Yuanzhi Li2016 · 8 citations
  85. SPRING: Studying Papers and Reasoning to play Games ↗neural information processing systems · no summary yetYue Wu et al.2023 · 6 citations
  86. The Mechanism of Prediction Head in Non-Contrastive Self-Supervised Learning ↗neural information processing systems · no summary yetZixin Wen, Yuanzhi Li2022 · 6 citations
  87. Local Signal Adaptivity: Provable Feature Learning in Neural Networks Beyond Kernels ↗neural information processing systems · no summary yetStefani Karp et al.2021 · 6 citations
  88. Improved Path-length Regret Bounds for Bandits ↗arXiv (Cornell University) · no summary yetSébastien Bubeck et al.2019 · 6 citations
  89. Provably Learning a Multi-head Attention Layer ↗ACM Symposium on Theory of Computing (STOC) · no summary yetSitan Chen, Yuanzhi Li2025 · 5 citations
  90. AgentKit: Structured LLM Reasoning with Dynamic Graphs ↗arXiv (Cornell University) · no summary yetYue Wu et al.2024 · 5 citations
  91. How Does Adaptive Optimization Impact Local Neural Network Geometry? ↗neural information processing systems · no summary yetKaiqi Jiang, Dhruv Malik, Yuanzhi Li2023 · 5 citations
  92. Learning (Very) Simple Generative Models Is Hard ↗arXiv (Cornell University) · no summary yetSitan Chen, Jerry Li, Yuanzhi Li2022 · 5 citations
  93. Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs ↗arXiv (Cornell University) · no summary yetSitan Chen et al.2022 · 5 citations
  94. Non-Stochastic Multi-Player Multi-Armed Bandits: Optimal Rate With Collision Information, Sublinear Without ↗arXiv (Cornell University) · no summary yetSébastien Bubeck et al.2019 · 5 citations
  95. Mixture of Parrots: Experts improve memorization more than reasoning ↗arXiv (Cornell University) · no summary yetSamy Jelassi et al.2024 · 4 citations
  96. LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch ↗arXiv (Cornell University) · no summary yetZhengzhong Liu et al.2025 · 3 citations
  97. Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoning ↗Proceedings of the AAAI Conference on Artificial Intelligence · no summary yetRuiqian Nai et al.2024 · 3 citations
  98. The Benefits of Mixup for Feature Learning ↗arXiv (Cornell University) · no summary yetDifan Zou et al.2023 · 3 citations
  99. Settling the Horizon-Dependence of Sample Complexity in Reinforcement Learning ↗Annual Symposium on Foundations of Computer Science · no summary yetYuanzhi Li, Ruosong Wang, Lin Forrest Yang2021 · 3 citations
  100. Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity ↗arXiv (Cornell University) · no summary yetDhruv Malik et al.2021 · 3 citations

1 2

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/yuanzhi-li/works/