🧑‍🔬 Arthur Mensch

Arthur Mensch works on large neural models for language and vision. The papers here, both from 2022, ask how such models should be built and trained. One looks at how model size and training data should grow together under a limited compute budget. The other looks at how a single model can handle images, video and text without task-specific fine-tuning. The methods are large-scale empirical ones: many models are trained and compared to find rules that guide practice. The work sits within Language models & pretraining and Vision & language.

The most cited of these papers is Flamingo: a Visual Language Model for Few-Shot Learning. It presents a family of visual language models that learn new image and video tasks from a handful of examples placed in the prompt. Unlike contrastive image–text systems, which can only rate how well a caption matches a picture, these models write open-ended text. The second paper, Training Compute-Optimal Large Language Models, studies how a fixed amount of compute should be shared between parameter count and training tokens. Three separate estimation methods point to the same answer: the two should be scaled in equal proportion. The study concludes that many language models of the time were oversized and trained on too little text. Both papers are also the most recent ones here, and together they show the work moving between efficient language-model scaling and multimodal few-shot learning.

The papers are described as coming from DeepMind teams. Bylines from 2021 and 2022 also list the Centre National de la Recherche Scientifique. The most frequent co-author here is Eliza Rutherford. Other collaborators include Jean-Baptiste Alayrac, Sebastian Borgeaud and Johannes Welbl.

Arthur Mensch has 2 papers in our library, in 2022, most of them on Language models & pretraining, Vision & language; often with Eliza Rutherford, Aidan Clark, Jean-Baptiste Alayrac.