Michiel de Jong
Michiel de Jong has 2 papers in our library, in 2023; often with James Lee-Thorp, Joshua Ainslie, Sumit Sanghai.
OpenAlex · ORCID · h-index 9 · 48 works · 641 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 636 citations in our library · ⭐ Paper Star rank #2073
Affiliations
🏢GoogleMountain View ·
United States2021–2024 · 11 papers🎓University of Southern CaliforniaLos Angeles ·
United States2018–2023 · 10 papers🎓Carnegie Mellon UniversityPittsburgh ·
United States2018–2021 · 3 papers🔬Netherlands Defence AcademyBreda ·
Netherlands2015–2021 · 3 papers🎓Windesheim University of Applied SciencesZwolle ·
Netherlands2019–2021 · 3 papers🎓University of TwenteEnschede ·
Netherlands2018–2019 · 2 papers🏥Atrium Medisch Centrum ParkstadHeerlen ·
Netherlands2014 · 2 papers
Concepts, shelves and topics
💡 Concepts 1 Transformer and attention · 1 work
📚 Shelves 5 Arts and Humanities · 2 works Computer Science · 2 works Artificial Intelligence · 2 works Computer Vision and Pattern Recognition · 2 works Language and Linguistics · 2 works
Works by year
2020s
Papers
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints (2023 · 1,012 citations)
During autoregressive decoding, a Transformer model loads the attention keys and values from memory at every step. - CoLT5: Faster Long-Range Transformers with Conditional Computation (2023 · 83 citations)
Encoding long documents with Transformers is costly. Sparse attention reduces the quadratic attention cost, but the feedforward and projection layers still run on every token and dominate compute in larger models. CoLT5 (Conditional LongT5), from Google Research, extends LongT5 with conditional computation. Each…
Co-authors here
- James Lee-Thorp ×2
- Joshua Ainslie ×2
- Sumit Sanghai ×2
- Yury Zemlyanskiy ×2
- David Uthus
- Mandy Guo
- Santiago Ontañón
- Siddhartha Brahma
- Tao Lei
- Yi Tay
- Yun-Hsuan Sung
Venues
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/michiel-de-jong/