Yuval Ran-Milo
Yuval Ran-Milo has 2 papers in our library, in 2026, most of them on Language models & pretraining, Interpretability.
No OpenAlex profile matched yet.
📄 2 works in our library · 🔗 Cited 1 time in our library · ⭐ Paper Stars ranking: #11763
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Artificial Intelligence · 2 worksArts and Humanities · 2 worksComputer Science · 2 worksLanguage and Linguistics · 2 works
🧭 Topics 2Research subjects and languages the work is about.Interpretability · 1 workLanguage models & pretraining · 1 work
Works by year
2020s
Papers
- Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks (2026 · 1 citation)
Softmax Transformers often put almost all attention on one fixed, content-independent position, usually BOS. - A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation (2026)
Transformers often send a large share of attention to the first token regardless of its content. This "attention sink" hurts context use, quantization, streaming and interpretability. The authors trace how the sink is produced inside the 124M-parameter GPT-2, which uses learned query biases and absolute positional…
Venues
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/yuval-ran-milo/