🧑‍🔬 William Fedus

William Fedus works on large language models: how they are trained, how they scale, and how their outputs can be evaluated. Their papers here ask how finetuning on instructions helps models handle tasks they were never shown, whether the inductive bias of an architecture changes its scaling behaviour, and how the factual accuracy of model answers can be measured reliably. The approach is large-scale and empirical. It involves training and finetuning models across many sizes, and building benchmarks with careful human checking. The work belongs to the area of Language models & pretraining.

The most cited of these papers, Scaling Instruction-Finetuned Language Models (2022), finetunes pretrained models such as T5, PaLM and U-PaLM on a broad mixture of tasks phrased as instructions. It then examines how results shift when more tasks are added, when models grow larger, and when examples with step-by-step reasoning are mixed in. Scaling Laws vs Model Architectures (2023) questions the habit of judging a new architecture at a single size. It tests whether gains seen at one scale hold at others, both during pretraining and once models are finetuned.

The newest work turns to the reliability of what models say. Measuring short-form factuality in large language models (2024), written with authors from OpenAI, presents SimpleQA. This is a benchmark of brief questions, each with one checkable answer. The questions were written adversarially against existing models and confirmed by a second trainer and by independent web sources. A prompted grader then sorts every model reply into one of three classes: right, wrong, or declined.

Fedus is currently affiliated with the Massachusetts Institute of Technology. Earlier bylines list Google, the Université de Montréal and Mila - Quebec Artificial Intelligence Institute. Frequent co-authors include Hyung Won Chung, Mostafa Dehghani and Yi Tay.

William Fedus has 3 papers in our library, in 2022–2024, most of them on Language models & pretraining; often with Hyung Won Chung, Mostafa Dehghani, Yi Tay.