Elamur Research⌕
☰
home / authors / Neil Houlsby

Neil Houlsby

Neil Houlsby has 3 papers in our library, in 2020–2022; often with Basil Mustafa, Joan Puigcerver, Mostafa Dehghani.

3 papers in our library · 2020–2022

OpenAlex · h-index 37 · 89 works · 30,402 citations OpenAlex, 2026-10-14

📄 3 works in our library · 🔗 Cited 7,255 times in our library · 6 more in our library without a page yet · ⭐ Paper Stars ranking: #100

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

 Affiliations

🏢GoogleMountain View · United States2018–2024 · 35 papers🎓University of CambridgeCambridge · United Kingdom2012–2021 · 13 papers🏢GoogleZurich · Switzerland2015–2022 · 5 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 2 more institutions on single bylines are not listed.

 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 3 worksArtificial Intelligence · 3 worksComputer Vision and Pattern Recognition · 3 worksArts and Humanities · 1 workLanguage and Linguistics · 1 work

💡 Concepts 1Key ideas and methods explained on our concept pages.Transformer and attention · 1 work

 Works by year

2020s

20221 paper20211 paper20201 paper

 Papers

The most cited first (OpenAlex and our library).

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020 · 23,726 citations)
    In 2020, image recognition still relied mainly on convolutional networks. Earlier efforts to use self-attention in vision either kept a CNN backbone or used custom attention patterns that ran slowly on modern accelerators. The authors ask whether an almost unchanged NLP-style Transformer encoder can classify images…
  2. Scaling Vision with Sparse Mixture of Experts (2021 · 128 citations)
    Large vision networks are usually dense: each parameter touches every input, so growing capacity drives up compute.
  3. Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints (2022 · 98 citations)
    Training large neural networks from random initialization is very expensive, so most practitioners keep reusing a small set of dense models.

All works

9 works of theirs in our library, the most cited first; 3 with a page here, the others link their original until we summarise them.

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScalearXiv:2010.11929Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al.2020 · 23,726 citations
  2. MLP-Mixer: An all-MLP Architecture for Vision ↗arXiv (Cornell University) · no summary yetIlya Tolstikhin et al.2021 · 1,513 citations
  3. Scaling Vision Transformers ↗IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · no summary yetXiaohua Zhai et al.2022 · 1,005 citations
  4. Self-Supervised GANs via Auxiliary Rotation Loss ↗IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · no summary yetTing Chen et al.2019 · 320 citations
  5. Scaling Vision Transformers to 22 Billion Parameters ↗arXiv:2302.05442 · no summary yetMostafa Dehghani et al.2023 · 217 citations
  6. Scaling Vision with Sparse Mixture of ExpertsarXiv:2106.05974Carlos Riquelme, Joan Puigcerver, Basil Mustafa, et al.2021 · 128 citations
  7. Sparse Upcycling: Training Mixture-of-Experts from Dense CheckpointsarXiv:2212.05055Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al.2022 · 98 citations
  8. The Benchmark Lottery ↗arXiv:2107.07002 · no summary yetMostafa Dehghani et al.2021 · 50 citations
  9. A Filtering Approach to Stochastic Variational Inference ↗no summary yetNeil Houlsby, David M. Blei2015 · 3 citations

 Co-authors here

12 of 18 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

 Cited by

Papers of our library cite their works 7,255 times (of the 149,439 papers whose bibliographies we read); 1,136 citing papers with a page here →.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/neil-houlsby/