🧑‍🔬 Suvrit Sra

🏛️ Affiliations

🎓Technical University of MunichMunich · Germany2024–2026 · 8 paperscurrent🎓Massachusetts Institute of TechnologyCambridge · United States2014–2023 · 99 papersEmployment since 2015🔬Max Planck SocietyMunich · Germany2003–2015 · 34 papers🔬Max Planck Institute for Biological CyberneticsTübingen · Germany2003–2014 · 33 papers🔬Max Planck Institute for Intelligent SystemsStuttgart · Germany2011–2015 · 25 papers🎓The University of Texas at AustinAustin · United States2003–2008 · 14 papersEducation🎓Umeå UniversityUmeå · Sweden2021–2025 · 6 papers🎓Carnegie Mellon UniversityPittsburgh · United States2014–2015 · 6 papers🎓Tsinghua UniversityBeijing · China2021–2023 · 3 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 2 more institutions on single bylines are not listed.

🧭 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📅 Works by year

2020s

📄 Papers

The most cited first (OpenAlex and our library).

1. Transformers learn to implement preconditioned gradient descent for in-context learning (2023 · 37 citations)
   The paper asks whether transformers merely can represent optimization algorithms or whether ordinary training actually drives them to do so.
2. Linear attention is (maybe) all you need (to understand transformer optimization) (2023 · 10 citations)
   Training Transformers is notoriously hard. Plain SGD lags well behind adaptive optimizers such as Adam, and the loss landscape shows unusual properties that classical optimization theory does not account for. Full Transformers are too complex to analyze directly, so the authors look for the simplest model that still…

All works

4 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

1. Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization ↗neural information processing systems · no summary yetSashank Jakkam Reddi et al.2016 · 96 citations
2. A Generic Approach for Escaping Saddle points ↗arXiv (Cornell University) · no summary yetSashank Jakkam Reddi et al.2017 · 52 citations
3. Transformers learn to implement preconditioned gradient descent for in-context learningConference on Neural Information Processing Systems (NeurIPS)Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, et al.2023 · 37 citations
4. Linear attention is (maybe) all you need (to understand transformer optimization)arXiv:2310.01082Kwangjun Ahn, Xiang Cheng, Minhak Song, et al.2023 · 10 citations

🤝 Co-authors here

3 co-authors with a page, the most papers together first.

• Kwangjun Ahn ×2
• Xiang Cheng ×2
• Chulhee Yun

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

• Conference on Neural Information Processing Systems (NeurIPS): NeurIPS 2023
• neural information processing systems

🔗 Cited by

Papers of our library cite their works 41 times (of the 149,439 papers whose bibliographies we read); 12 citing papers with a page here →.

Updated 3 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/suvrit-sra/