Elamur Research⌕
☰
home / papers / 2026 / EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task

EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task

Paper · 2026 · Elio Musacchio, Lucia Siciliani, Pierpaolo Basile et al.

Summary

EVWSD-ITA is a shared task at EVALITA 2026 that extends Visual Word Sense Disambiguation (VWSD) to fine-grained distinctions in Italian. In the original SemEval 2023 VWSD setup, wrong candidate images usually belong to unrelated senses, so coarse semantic matching is often enough. In EVWSD-ITA, a system receives a short query with a target word and context words. It must identify the single correct image among ten candidates. The distractors mix two kinds of images: other senses of the same lemma, and "hard negatives". Hard negatives are co-hyponyms that share the target's hypernym, such as a harpsichord instead of a piano.

The data come from BabelNet synsets, their glosses and their "best image". The organizers provide a large automatically built train set. The test set was validated by hand to remove near-identical senses, visually indistinguishable images and abstract concepts, and its queries were written manually.

Only one team, UniTor, participated. Its pipeline combined LLM-generated questions, VQA confidence and CLIP similarity, and it clearly outperformed the multilingual CLIP baseline. The results suggest that multimodal LLMs help with fine-grained visual sense distinctions.

Key points

  • Task format: each query has ten candidate images and exactly one correct answer. There are two separate leaderboards, HIT@1 and MRR, because systems that only predict an answer cannot produce the ranking MRR requires.
  • Dataset construction: the data are built from BabelNet.
    • One gloss is chosen per synset, with sources prioritized in this order: WordNet, Wiktionary, Wikipedia, Wikidata, then other sources.
    • Each query combines the target lemma, a lemma of its hypernym and a word from the gloss.
    • After filtering, test instances with fewer than ten candidates were filled up with random synsets taken from other instances.
  • Baseline: zero-shot clip-ViT-B-32-multilingual-v1 (sentence-transformers), ranking images by cosine similarity, scored HIT@1 .4505 and MRR .6244.
  • Best system: UniTor submitted three runs.
    • Its best run, Score VQA + CLIP (Query) + CLIP (Description), reached HIT@1 .7117 and MRR .8182.
    • The pipeline used GPT-5.1 to generate questions and Qwen3-VL Yes/No answer probabilities for VQA scoring.
  • Other UniTor runs: the run without the description score achieved MRR .8100. The stricter "Two-Step" question parsing variant scored lowest (.6667 HIT@1 / .7770 MRR) but still beat the baseline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 worksWord sense disambiguation · 87 works

Cite

Cite as Elio Musacchio, Lucia Siciliani, Pierpaolo Basile and Giovanni Semeraro. 2026. EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{musacchio2026evwsd,
  title = {{EVWSD-ITA at EVALITA 2026: Overview of the Enhanced Visual Word Sense Disambiguation for Italian Task}},
  author = {Elio Musacchio, Lucia Siciliani, Pierpaolo Basile and Giovanni Semeraro},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.47/}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Navigli, Word sense disambiguation: A survey, ACM computing surveys (CSUR) 41 (2009) 1–69. find
  2. M. Bevilacqua, T. Pasini, A. Raganato, R. Navigli, Recent trends in word sense disambiguation: A survey, in: International joint conference on artificial intelligence, International Joint Conference on Artificial Intelligence, Inc, 2021, pp. 4330–4338. find
  3. D. Sumanathilaka, N. Micallef, J. Hough, Assessing gpt’s potential for word sense disambiguation: A quantitative evaluation on prompt engineering techniques, in: 2024 IEEE 15th Control and System Graduate Research Colloquium (ICSGRC), IEEE, 2024, pp. 204–209. find
  4. A. Raganato, I. Calixto, A. Ushio, J. Camacho-Collados, M. T. Pilehvar, SemEval-2023 task 1: Visual word sense disambiguation, in: A. K. Ojha, A. S. Doğruöz, G. Da San Martino, H. Tayyar Madabushi, R. Kumar, E. Sartori (Eds.), Proceedings of the 17th International Workshop on Semantic Evaluation (Se… find
  5. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  6. T. Pasini, A. Raganato, R. Navigli, Xl-wsd: An extra-large and cross-lingual evaluation framework for word sense disambiguation, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 13648–13656. find
  7. L. Huang, C. Sun, X. Qiu, X.-J. Huang, Glossbert: Bert for word sense disambiguation with gloss knowledge, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 35…
  8. J. H. Yae, N. C. Skelly, N. C. Ranly, P. M. LaCasse, Leveraging large language models for word sense disambiguation, Neural Computing and Applications 37 (2025) 4093–4110. find
  9. D. Meconi, S. Stirpe, F. Martelli, L. Lavalle, R. Navigli, Do large language models understand word senses?, in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 33885–33904.
  10. A. Kritharoula, M. Lymperaiou, G. Stamou, Large language models and multimodal retrieval for visual word sense disambiguation, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, S… find
  11. S. Kwon, R. Garodia, M. Lee, Z. Yang, H. Yu, Vision meets definitions: Unsupervised visual word sense disambiguation incorporating gloss information, in: A. Rogers, J. Boyd-Graber, N. Okazaki (Eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L… find
  12. X. Zhang, T. Zhen, J. Zhang, Y. Wang, S. Liu, Srcb at semeval-2023 task 1: Prompt based and cross-modal retrieval enhanced visual word sense disambiguation, in: Proceedings of the 17th international workshop on semantic evaluation (SemEval-2023), 2023, pp. 439–446. find
  13. Q. Yang, X. Wang, Y. Li, L.-K. Lee, F. L. Wang, T. Hao, Mta: A lightweight multilingual text alignment model for cross-language visual word sense disambiguation, in: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 12166–12170. find
  14. Y. Laba, Y. Mohytych, I. Rohulia, H. Kyryleyza, H. Dydyk-Meush, O. Dobosevych, R. Hryniv, Ukrainian visual word sense disambiguation benchmark, in: Proceedings of the Third Ukrainian Natural Language Processing Workshop (UNLP)@ LREC-COLING 2024, 2024, pp. 61–66. find
  15. I. Setitra, P. Rajapaksha, A. K. Myat, N. Crespi, Leveraging ensemble deep models and llm for visual polysemy and word sense disambiguation, Multimedia Tools and Applications (2025) 1–33. find
  16. E. Musacchio, L. Siciliani, P. Basile, G. Semeraro, Assessing and improving the multilingual visual word sense disambiguation ability of vision-language models, in: ECAI 2025, IOS Press, 2025, pp. 4145 – 4152. find
  17. J. Robinson, C.-Y. Chuang, S. Sra, S. Jegelka, Contrastive learning with hard negative samples, arXiv preprint arXiv:2010.04592 (2020). — [arXiv:2010.04592](https://arxiv.org/abs/2010.04592) find
  18. R. Navigli, S. P. Ponzetto, Babelnet: Building a very large multilingual semantic network, in: Proceedings of the 48th annual meeting of the association for computational linguistics, 2010, pp. 216–225.
  19. N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 2019. URL: http://arxiv.org/abs/1908.10084. ↗
  20. C. D. Hromei, A. Scaiella, D. Croce, R. Basili, Unitor at evwsd-ita: Zero-shot visual word sense disambiguation via visual question-answering, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/musacchio-2026-evwsd-ita-evalita-2026-overview/