SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
Paper · 2025 · Michael Tschannen, Alexey Gritsenko, Xiao Wang et al.
Summary
SigLIP 2 is Google DeepMind's second generation of open image-text encoders. CLIP-style models do well at zero-shot classification and retrieval, but they are weaker at localization and dense per-patch features. Earlier open checkpoints also rarely combine recent training advances. The authors merge several previously separate techniques into one staged recipe built on SigLIP's sigmoid loss:
- a LocCa-style decoder trained on captioning, grounded captioning and referring expressions;
- SILC/TIPS self-distillation and masked prediction, added only for the last 20% of training;
- curation-based distillation fine-tuning for the smallest models;
- a NaFlex variant that keeps native aspect ratios and supports several sequence lengths.
Training uses WebLI with a 90/10 English/non-English mix, de-biasing filters and the multilingual Gemma tokenizer. At every model size, SigLIP 2 outperforms SigLIP on zero-shot classification, retrieval, VLM transfer, dense prediction and localization. It also improves multilingual, cultural-diversity and fairness metrics. The architecture is unchanged, so existing users only need to swap in the new weights and tokenizer.
Key points
- Checkpoints come in four sizes: ViT-B (86M), L (303M), So400m (400M) and g (1B), plus NaFlex variants. The decoder used during training is not part of the release.
- Training data is WebLI (10B images, 12B alt-texts, 109 languages). The run covers 40B examples with batch size 32k on up to 2048 TPUv5e chips. Text uses a 256k-vocabulary Gemma tokenizer with a 64-token length.
- On referring expression comprehension (RefCOCO, RefCOCO+, RefCOCOg, Acc@0.5), SigLIP 2 clearly beats SigLIP. Only LocCa, which was trained on English-only captions, scores higher.
- In cultural-diversity tests with L/16 at 256px, GeoDE 10-shot region geolocalization rises from 36.2% to 44.4%. Dollar Street 0-shot accuracy rises from 52.1% to 55.2%.
- Representation bias for L/16 at 256px drops from about 35.5% (SigLIP) to 7.3%, and larger models tend to show less bias.
Fields, topics and concepts
📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works
🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 882 worksVision & language · 811 works
Cite
Cite as Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Henaff, Jeremiah Harmsen, Andreas Steiner and Xiaohua Zhai. 2025. SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. arXiv:2502.14786.
@misc{tschannen2025siglip,
title = {{SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features}},
author = {Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Henaff, Jeremiah Harmsen, Andreas Steiner and Xiaohua Zhai},
year = {2025},
doi = {10.48550/arxiv.2502.14786},
eprint = {2502.14786},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.2502.14786}
}References
- [1] I. Alabdulmohsin, X. Zhai, A. Kolesnikov, and L. Beyer. Getting vit in shape: Scaling laws for compute-optimal model design. In NeurIPS, 2023. find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/tschannen-2025-siglip-2-multilingual-vision-language/