Elamur Research⌕
☰
home / authors / Shaden Smith

Shaden Smith

Shaden Smith has 2 papers in our library, in 2021–2022, most of them on Language models & pretraining, Text generation, GPU & parallel computing; often with Samyam Rajbhandari, Brandon Norick, Jared Casper.

2 papers in our library · 2021–2022

OpenAlex · ORCID · h-index 22 · 39 works · 2,814 citations OpenAlex, 2026-10-14

📄 2 works in our library · 🔗 Cited 1,849 times in our library · 1 more in our library without a page yet · ⭐ Paper Stars ranking: #2972

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

 Affiliations

🏢MicrosoftRedmond · United States2020–2024 · 6 paperscurrent🎓University of MinnesotaMinneapolis · United States2014–2021 · 17 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 3 more institutions on single bylines are not listed.

 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 2 worksArtificial Intelligence · 2 worksHardware and Architecture · 2 worksArts and Humanities · 1 workLanguage and Linguistics · 1 workInformation Systems · 1 work

🧭 Topics 3Research subjects and languages the work is about.GPU & parallel computing · 1 workLanguage models & pretraining · 1 workText generation · 1 work

 Works by year

2020s

20221 paper20211 paper

 Papers

The most cited first (OpenAlex and our library).

  1. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022 · 615 citations)
    The paper describes how Microsoft and NVIDIA trained Megatron-Turing NLG 530B (MT-NLG), a dense, left-to-right autoregressive transformer with 530 billion parameters.
  2. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning (2021 · 254 citations)
    Dense model sizes have grown roughly a thousandfold in three years, while single-GPU memory has grown only about fivefold.

All works

3 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

  1. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model ↗arXiv:2211.05100 · no summary yetBigScience Workshop et al.2022 · 1,606 citations
  2. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language ModelarXiv:2201.11990Shaden Smith, Mostofa Patwary, Brandon Norick, et al.2022 · 615 citations
  3. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep LearningSC 2021Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, et al.2021 · 254 citations

 Co-authors here

11 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

 Cited by

Papers of our library cite their works 1,849 times (of the 149,439 papers whose bibliographies we read); 400 citing papers with a page here →.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/authors/shaden-smith/