🧑‍🔬 Aidan Clark

Aidan Clark's papers here concern large language models and the systems built on them, within the topic of Language models & pretraining. The questions are practical ones for anyone training or releasing such models. One is how a fixed amount of compute should be divided between model size and training data. Another is how a widely deployed model can be examined for risks before release. The methods range from large-scale training experiments to structured safety evaluation.

The most cited of the works is the GPT-4o System Card (2024). It reports how a single network handling text, audio, images and video was tested and its hazards reduced, with particular attention to the dangers that arise when the model converses by voice in real time. The earlier Training Compute-Optimal Large Language Models (2022) trained transformer models across a wide range of sizes and token counts. From several independent estimates, it concluded that parameters and training data should grow in step, so by that measure many language models of the period were oversized and undertrained. Read in order, the two papers move from how to train models efficiently toward assessing the safety of a multimodal system meant for public use.

Byline affiliations place Clark at Google and Google DeepMind between 2019 and 2022, the period of the DeepMind study on compute-optimal scaling. The system card appears under the collective authorship of OpenAI. Co-authors on Clark's papers here include Sebastian Borgeaud, Arthur Mensch and Alec Radford.

Aidan Clark has 2 papers in our library, in 2022–2024, most of them on Language models & pretraining; often with Adam Lerer, Adam Perelman, Aditya Ramesh.