Elamur Research⌕
☰
home / papers / 2026 / UWB at SemEval-2026 Task 5: Synsets and their contexts

UWB at SemEval-2026 Task 5: Synsets and their contexts

Paper · 2026 · Stephen Taylor

Summary

This system paper describes a University of West Bohemia entry to SemEval-2026 Task 5. The task asks systems to predict how plausible human annotators find a given WordNet sense of an ambiguous word within short, five-part stories. The stories are from the AmbiStory dataset. Each has three context sentences, a neutral sentence containing the homonym, and an optional ending.

The approach links each WordNet 3.0 synset to a fixed list of 50 related "bias words", inspired by Pilehvar and Collier (2016). It then derives features measuring how well each story segment fits that list:

  • Jaccard set overlaps,
  • rank estimates,
  • average embedding cosine distances,
  • a three-valued synset polarity.

Linear regression models, one per human annotator and split by polarity group, are averaged to produce the final score.

Results were close to chance. The author argues that the dataset deliberately mixes cues for competing senses, so topical word lists cannot separate them. The author also notes that some feature values point to code bugs. The paper is useful mainly as a documented negative result for related-word methods on this task.

Key points

  • Data: train.json has 2280 records, dev.json 588, test.json 930. Most stories appear in 6 variants covering two senses and different endings, each rated by 5 humans.
  • Model: scikit-learn linear regression with five per-annotator models averaged, plus 9 groups based on precontext/ending polarity (−1, 0, +1). Word vectors come from NLPL embedding #20.
  • Test results: 0.55 accuracy (prediction within one standard deviation of the human mean), Spearman 0.13, and a combined score of 0.34, placing 74th of 78 entrants.
  • The polarity feature reached 0.412 accuracy against gold values derived from the training data. One feature produced 956 invalid zero values, which suggests an implementation error.
  • The related work covers synset methods built on random walks and Personalized PageRank, as in Goikoetxea et al. (2015) and Agirre et al. (2014), along with the gloss vectors of Patwardhan and Pedersen (2006). Code is available at github.com/StephenETaylor/SE26-5.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of West Bohemia in PilsenPilsen · Czechia14 works · 38,760 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Stephen Taylor. 2026. UWB at SemEval-2026 Task 5: Synsets and their contexts. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.106

@inproceedings{taylor2026uwb,
  title = {{UWB at SemEval-2026 Task 5: Synsets and their contexts}},
  author = {Stephen Taylor},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.106},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.106}
}

References

18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Oier López de Lacalle, and Aitor Soroa. 2014. Random walks for knowledge-based word sense disambiguation. Computational Linguistics, 40(1):57–84.
  2. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc. find
  3. Murhaf Fares, Andrei Kutuzov, Stephan Oepen, and Erik Velldal. 2017. Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In Proceedings of the 21st Nordic Conference on Computational Linguistics (NoDaLiDa), pages 271–276, Gothenburg, Sweden. Linköping Univ… find
  4. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
  5. Janosch Gehring, Selina Meyer, and Michael Roth. 2026. SemEval-2026 task 5: Rating plausibility of word senses in ambiguous stories through narrative understanding. In Proceedings of the 20th International Workshop on Semantic Evaluation, San Diego, California. Association for Computational Linguist… find
  6. Janosch Gehring and Michael Roth. 2025. Ambistory: A challenging dataset of lexically ambiguous short stories. In Proceedings of the 14th Joint Conference on Lexical and Computational Semantics, pages 152– 171. Association for Computational Linguistics. find
  7. Josu Goikoetxea, Aitor Soroa, and Eneko Agirre. 2015. Random walks and neural network language models on knowledge bases. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1434–1439, Denver, Colora… find
  8. Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation, pages 24–26. find
  9. Rada Mihalcea and Dan I Moldovan. 2001. extended wordnet: Progress report. find
  10. George A. Miller. 1995. Wordnet: a lexical database for English. Communications of the (ACM), 38(11):39–41.
  11. Siddharth Patwardhan and Ted Pedersen. 2006. Using wordnet-based context vectors to estimate the semantic relatedness of concepts. In Proceedings of the Workshop on Making Sense of Sense: Bringing Psycholinguistics and Computational Linguistics Together. find
  12. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:282… find
  13. Mohammad Taher Pilehvar and Nigel Collier. 2016. De-conflated semantic representations. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1680–1690, Austin, Texas. Association for Computational Linguistics. find
  14. Alexandre Rademaker, Abhishek Basu, and Rajkiran Veluri. 2023. Semantic parsing and sense tagging the Princeton WordNet gloss corpus. In Proceedings of the 12th Global Wordnet Conference, pages 243– 253, University of the Basque Country, Donostia - San Sebastian, Basque Country. Global Wordnet Assoc…
  15. Alexandre Rademaker, Bruno Cuconato, Alessandra Cid, Alexandre Tessarollo, and Henrique Andrade. 2019. Completing the Princeton annotated gloss corpus project. In Proceedings of the 10th Global Wordnet Conference, pages 378–386, Wroclaw, Poland. Global Wordnet Association. find
  16. Herbert Rubenstein and John B. Goodenough. 1965. Contextual correlates of synonymy. Commun. ACM, 8(10):627–633. find
  17. Sanda Harabagiu and Dan Moldovan. 1998. Knowledge processing on an extended wordnet. WordNet: An electronic lexical database, 305:381–405. find
  18. Taher H. Haveliwala. 2002. Topic-sensitive pagerank. In WWW02: Hypermedia Track of the Elevnth International Word-Wide Web Conference. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/taylor-2026-uwb-semeval-2026-task-5/