🧑‍🔬 Sham M. Kakade

🏛️ Affiliations

🎓Harvard UniversityCambridge · United States2021–2026 · 51 paperscurrent🎓University of WashingtonSeattle · United States2015–2023 · 62 papers🏢MicrosoftRedmond · United States2009–2022 · 32 papers🎓University of PennsylvaniaPhiladelphia · United States2004–2018 · 25 papers🎓Toyota Technological Institute at ChicagoChicago · United States2005–2009 · 23 papers🏢Microsoft Research New EnglandCambridge · United States2009–2015 · 18 papers🔬Gatsby Computational Neuroscience UnitLondon · United Kingdom1999–2002 · 9 papers🎓University College LondonLondon · United Kingdom2000–2003 · 7 papers🏢AmazonSeattle · United States2018–2024 · 6 papers🏢Microsoft Research New York CityNew York · United States2021–2022 · 3 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 15 more institutions on single bylines are not listed.

🧭 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📅 Works by year

2020s

📄 Papers

The most cited first (OpenAlex and our library).

1. Repeat After Me: Transformers are Better than State Space Models at Copying (2024 · 57 citations)
   The paper asks what models with a constant-size memory give up in exchange for cheap inference. The authors call this family "generalized state space models" (GSSMs). It includes S4, Mamba, LSTMs and linear-attention variants. The authors find that the main loss is the ability to reproduce or retrieve arbitrary parts…
2. Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining (2025 · 34 citations)
   The paper asks why reinforcement-learning fine-tuning improves mathematical reasoning in language models. Pretraining data for popular base models such as Qwen is usually undisclosed, which makes it hard to separate the effect of RL from the effect of pretraining. To get around this, the authors train decoder-only…
3. Transcendence: Generative Models Can Outperform The Experts That Train Them (2024 · 12 citations)
   Models trained with cross-entropy learn to mimic the distribution of their training data. This suggests they cannot outperform the humans who produced that data. The authors examine this assumption and define "transcendence" as a generative model earning higher reward than every expert behind its training set.

All works

16 works of theirs in our library, the most cited first; 3 with a page here, the others link their original until we summarise them.

1. Cover trees for nearest neighbor ↗International Conference on Machine Learning · no summary yetAlina Beygelzimer, Sham M. Kakade, John C. Langford2006 · 825 citations
2. Multi-view clustering via canonical correlation analysis ↗International Conference on Machine Learning · no summary yetKamalika Chaudhuri et al.2009 · 791 citations
3. Multi-Label Prediction via Compressed Sensing ↗neural information processing systems · no summary yetJohn C. Langford et al.2009 · 255 citations
4. Repeat After Me: Transformers are Better than State Space Models at CopyingarXiv:2402.01032Samy Jelassi, David Brandfonbrener, Sham M. Kakade, et al.2024 · 57 citations
5. Identifiability and Unmixing of Latent Parse Trees ↗arXiv (Cornell University) · no summary yetDaniel Hsu, Sham M. Kakade, Percy Liang2012 · 38 citations
6. Echo Chamber: RL Post-training Amplifies Behaviors Learned in PretrainingarXiv:2504.07912Rosie Zhao, Alexandru Meterez, Sham Kakade, et al.2025 · 34 citations
7. Prediction with a short memory ↗ACM SIGACT Symposium on Theory of Computing (STOC) · no summary yetVatsal Sharan et al.2016 · 29 citations
8. LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks ↗Proceedings of the 31st International Conference on Computational Linguistics: Industry Track · no summary yetAkshara Prabhakar et al.2025 · 14 citations
9. Transcendence: Generative Models Can Outperform The Experts That Train ThemConference on Neural Information Processing Systems (NeurIPS)Edwin Zhang, Vincent Zhu, Naomi Saphra, et al.2024 · 12 citations
10. Learning Overcomplete HMMs ↗arXiv (Cornell University) · no summary yetVatsal Sharan et al.2017 · 11 citations
11. Instabilities of Offline RL with Pre-Trained Neural Representation ↗arXiv (Cornell University) · no summary yetRuosong Wang et al.2021 · 9 citations
12. Mixture of Parrots: Experts improve memorization more than reasoning ↗arXiv (Cornell University) · no summary yetSamy Jelassi et al.2024 · 4 citations
13. Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models ↗arXiv (Cornell University) · no summary yetSamy Jelassi et al.2026 · 1 citation
14. Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass ↗neural information processing systems · no summary yetEthan Shen et al.2024 · 1 citation
15. Domain Adaptation: Overfitting and Small Sample Statistics ↗arXiv (Cornell University) · no summary yetDean P. Foster, Sham M. Kakade, Ruslan Salakhutdinov2011 · 1 citation
16. Domain Adaptation: A Small Sample Statistical Approach ↗International Conference on Artificial Intelligence and Statistics · no summary yetRuslan Salakhutdinov, Sham M. Kakade, Dean P. Foster2012

🤝 Co-authors here

5 co-authors with a page, the most papers together first.

• Eran Malach ×3
• Samy Jelassi ×2
• Benjamin L. Edelman
• David Brandfonbrener
• Naomi Saphra

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

• Conference on Neural Information Processing Systems (NeurIPS) ×2: NeurIPS 2024
• International Conference on Machine Learning ×2
• neural information processing systems ×2
• ACM SIGACT Symposium on Theory of Computing (STOC)
• International Conference on Artificial Intelligence and Statistics
• Proceedings of the 31st International Conference on Computational Linguistics: Industry Track

🔗 Cited by

Papers of our library cite their works 335 times (of the 149,439 papers whose bibliographies we read); 97 citing papers with a page here →.

Updated 3 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/sham-m-kakade/