🧑‍🔬 Xiaohua Zhai

Xiaohua Zhai works on large-scale visual representation learning, with papers here centred on two linked questions: whether architectures built for language can serve image recognition with little change, and how images and text can be trained together efficiently at scale. The work typically treats engineering constraints, such as memory and communication across many devices, as part of the research problem. It sits within the wider area of Language models & pretraining.

The most cited of Zhai's papers here is An Image is Worth 16x16 Words (2020), which introduced the Vision Transformer. The model cuts an image into fixed-size patches, projects each into a vector and feeds the resulting sequence to an almost unmodified Transformer encoder. It asks whether convolutional networks are really needed for classification, and it notes that without vision-specific assumptions such as locality the model lags a comparable ResNet when trained on ImageNet alone. The second paper, Sigmoid Loss for Language Image Pre-Training (2023), revisits contrastive image-text training of the CLIP and ALIGN kind. It swaps the batch-wide softmax for a pairwise sigmoid objective that treats every image-text pair as its own binary decision.

The 2023 sigmoid-loss paper shows where the work has moved: from image-only models towards joint vision-language pre-training. Because the new loss needs no global normalisation, it can be computed in small blocks spread across devices, and this keeps memory use low even for large batches. Zhai is currently at Google and Google DeepMind, and earlier worked at Google in Zurich. Frequent co-authors include Lucas Beyer and Alexander Kolesnikov, along with collaborators such as Mostafa Dehghani, Thomas Unterthiner, Jakob Uszkoreit and Basil Mustafa.

Xiaohua Zhai has 2 papers in our library, in 2020–2023, most of them on Language models & pretraining; often with Alexander Kolesnikov, Lucas Beyer, Basil Mustafa.