Elamur Research⌕
☰
home / papers / 2026 / A Large-Scale Instruction-Tuning Dataset and Models for Slovenian Vision-Language Tasks

A Large-Scale Instruction-Tuning Dataset and Models for Slovenian Vision-Language Tasks

Summary

Most vision-language models (VLMs) are trained on English data, so languages like Slovenian, with roughly 2.5 million speakers, have very few paired image-text resources. The authors address this gap with SLO-VLM-IT-Dataset 1.0, the first broad instruction-tuning collection for Slovenian VLMs, organised as dialogues. It has two sources. Automatically harvested material from Slovenian news portals and Slovenian Wikipedia provides local cultural context. A machine-translated version of the English LLaVA-665k set adds a wide range of instruction-following tasks. The authors used this resource to fine-tune multilingual Gemma-3 models with 4B and 12B parameters and released them as SVILA. They evaluated the models on a manually checked Slovenian test set, with an LLM judge also scoring the answers. SVILA clearly outperforms the original Gemma-3 checkpoints on visual question answering, visual grounding and OCR. The work gives Slovenian openly available instruction-tuned VLMs. It also offers a reusable recipe for other less-resourced languages.

Key points

  • Dataset and availability: SLO-VLM-IT-Dataset 1.0 contains 1,128,228 examples (over one million text-image pairs). It is published on CLARIN.SI at hdl.handle.net/11356/2050.
  • Models and code: SVILA-1-4B and SVILA-1-12B are hosted on Hugging Face under GaMS-Beta. They are built on Gemma-3, whose SigLIP vision encoder stays frozen during training. The training code is at github.com/matejMartinc/SVILA.
  • Test set curation: Annotators corrected machine-translated items task by task across four separate files, including 1000 general VQA and 1000 OCR examples. They discarded items built on puns or culturally unlocalizable content. In OCR items, the expected answers stay in the source language to test cross-lingual text extraction.
  • LLM-judge evaluation: Each answer receives a score from 1 to 5 on four criteria: how faithful it is to the image, how relevant and complete it is, how grammatically correct and fluent its language is, and how concise and clear it is. The judge then names the best model.
  • Context: Existing Slovenian-relevant resources (WIT, GLAMI-1M, the CLARIN.SI multimodal corpus) are too sparse, too narrow or not image-text data. The Slovenian GaMS LLMs from the PoVeJMo project are text-only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2e3jf6e7tcoh

@inproceedings{martinc2026large,
  title = {{A Large-Scale Instruction-Tuning Dataset and Models for Slovenian Vision-Language Tasks}},
  author = {Martinc, Matej and Vreš, Domen},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2e3jf6e7tcoh},
  url = {https://doi.org/10.63317/2e3jf6e7tcoh}
}

References

26 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. find our card
  2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736. find our card
  3. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607–15631, Toronto, Canada. Association for Computational Linguistics. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv prep… find our card
  5. Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530. find our card
  6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  7. Špela Arhar Holdt, Špela Antloga, Tina Munda, Eva Pori, and Simon Krek. 2025. From words to action: A national initiative to overcome data scarcity for the Slovene LLM. In Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025),… find
  8. Ashhadul Islam, Md Rafiul Biswas, Wajdi Zaghouani, Samir Brahim Belhaouari, and Zubair Shah. 2023. Pushing boundaries: Exploring zero shot object classification with large multimodal models. In 2023 Tenth International Conference on Social Networks Analysis, Management and Security (SNAMS), pages 1–… find
  9. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR. find our card
  10. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  11. Vaclav Kosar, Antonín Hoskovec, Milan Šulc, and Radek Bartyzal. 2022. GLAMI-1M: A multilingual image-text fashion dataset. arXiv preprint arXiv:2211.14451. — [arXiv:2211.14451](https://arxiv.org/abs/2211.14451) find
  12. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR. find our card
  13. Qiaomu Li, Ying Xie, Nina Grundlingh, Varsha Rani Chawan, and Cody Wang. 2024. Assessing image-captioning models: A novel framework integrating statistical analysis and metric patterns. In Proceedings of the Seventh Workshop on e-Commerce and NLP @ LREC-COLING 2024, pages 79–87, Torino, Italia. ELRA… find
  14. Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. 2025. A survey of state of the art large vision language models: Alignment, benchmark, evaluations and challenges. arXiv preprint arXiv:2501.02189. — [arXiv:2501.02189](https://arxiv.org/abs/2501.02189) find
  15. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306. find our card
  16. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916. find our card
  17. Marian Lupascu, Ana-Cristina Rogoz, Mihai Sorin Stupariu, and Radu Tudor Ionescu. 2025. Large multimodal models for low-resource languages: a survey. arXiv preprint arXiv:2502.05568. find
  18. Nitesh Patnaik, Navdeep Nayak, Himani Bansal Agrawal, Moinak Chinmoy Khamaru, Gourav Bal, Saishree Smaranika Panda, Rishi Raj, Vishal Meena, and Kartheek Vadlamani. 2025. Small vision-language models: A survey on compact architectures and techniques. arXiv preprint arXiv:2503.10665. — [arXiv:2503.10665](https://arxiv.org/abs/2503.10665) find
  19. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. Pm… find our card
  20. Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556–2565,… find our card
  21. Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Marc Najork. 2021. WIT: Wikipedia-Based Image Text Dataset for Multimodal Multilingual Machine Learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’2… find
  22. Domen Vreš, Martin Božič, Aljaž Potočnik, Tomaž Martinčič, and Marko Robnik-Šikonja. 2024. Generative model for less-resourced language with 1 billion parameters. In Proceedings of the Language technologies and digital humanities conference 2024, Ljubljana, Slovenia. find
  23. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. find our card
  24. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986. 11. find our card
  25. Language Resource References Kosar, Vaclav and Hoskvec, Antonín and Šulc, Milan and Bartyzal, Radek. 2022. GLAMI-1M: A Multilingual Image-Text Fashion Dataset. Zenodo. PID https://doi.org/10.5281/zenodo.73387922. — [doi:10.5281/zenodo.73387922](https://doi.org/10.5281/zenodo.73387922) ↗
  26. Izidor Mlakar, Simona Majhenič, Matej Rojc, and Darinka Verdonik. 2020. Multimodal corpus EVA 1.0. Slovenian language resource repository CLARIN.SI. 12. Appendices find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/martinc-2026-large-scale-instruction-tuning-dataset/