Elamur Research⌕
☰
home / papers / 2026 / ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD

ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD

Paper · 2026 · Sunny Zhou, Jordan Youner and Dean Cahill

Summary

The paper describes the ZYC team's system for SemEval-2026 Task 5. The task asks for a 1–5 plausibility rating of a given sense of an ambiguous word within a short narrative, and the ratings should agree with human annotators. The authors use the AmbiStory dataset and merge each story's precontext, ambiguous sentence and optional ending into one "full context." They treat scoring as five-way classification and compare four methods built on BERT-like encoders. The first is a naive cosine-similarity baseline. The second (CCE) pools definition embeddings with SBERT and feeds the similarity scores to a classifier. The other two use cross-attention inspired by GLiNER and trained from scratch: one attends over definitions (DXA) and the other over WordNet synonyms (SER). Their strongest test-set result reached 57% accuracy, alongside a Spearman score of 0.20.

The pretrained backbones produced similar scores within each architecture. The simpler CCE outperformed the attention-based models. The authors conclude that the fine-tuning procedure and the training curriculum matter more than the choice of pretrained model for this new task.

Key points

  • Data: AmbiStory provides 2279 train, 587 dev and 929 test instances. The authors recombined train fields into 12,159 pairs for MLM-style pretraining of DXA.
  • Evaluation: A prediction counts as correct if it falls within one standard deviation of the annotators' mean. The system's prediction is the expected value of its class distribution.
  • Training and SER design:
    • DXA and SER train the attention head and the classifier separately.
    • The classifier is trained with gradual unfreezing, following ULMFiT.
    • SER represents senses as WordNet synonyms separated by [SYN] tokens, plus a trainable [MISC] token for senses WordNet does not cover.
  • Overfitting: Early DXA runs overfit badly, reaching about 0.98 train accuracy against 0.4 on validation.
  • Variance and Spearman: Analysis showed that higher variance (coefficient of variation) in similarity scores went with lower Spearman. CCE-DistilRoBERTa had the highest variance and a negative Spearman.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Word sense disambiguation · 83 works

Cite

Cite as Sunny Zhou, Jordan Youner and Dean Cahill. 2026. ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.371

@inproceedings{zhou2026zyc,
  title = {{ZYC at SemEval-2026 Task 5: Application of BERT-based Contextual Embeddings Similarity for WSD}},
  author = {Sunny Zhou, Jordan Youner and Dean Cahill},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.371},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.371}
}

References

14 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Janosch Gehring, Selina Meyer, and Michael Roth. 2026. SemEval-2026 task 5: Rating plausibility of word senses in ambiguous stories through narrative understanding. In Proceedings of the 20th International Workshop on Semantic Evaluation, San Diego, California. Association for Computational Linguist… find
  3. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. Preprint, arXiv:2006.03654.
  4. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia. Association for Computational Linguistics. find
  5. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuanjing Huang. 2019. GlossBERT: BERT for word sense disambiguation with gloss knowledge. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I…
  6. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. Preprint, arXiv:1908.10084.
  7. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Preprint, arXiv:2004.09297.
  8. Ryu Suzuki, Kanako Komiya, Masayuki Asahara, Minoru Sasaki, and Hiroyuki Shinnou. 2018. All words word sense disambiguation using concept embeddings. In Proceedings of the 11th International Conference on Language Resources and Evaluation, Miyazaki, Japan. European Language Resources Association. find
  9. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA. Cur…
  10. Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2024. GLiNER: Generalist model for named entity recognition using bidirectional transformer. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol…
  11. Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th Annual International Conference on Systems Documentation, SIGDOC ’86, page 24–26, New York, NY, USA. Association for Computing Machinery. find
  12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  13. Fuli Luo, Tianyu Liu, Zexue He, Qiaolin Xia, Zhifang Sui, and Baobao Chang. 2018. Leveraging gloss knowledge in neural word sense disambiguation by hierarchical co-attention. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find
  14. George A. Miller. 1994. WordNet: A lexical database for English. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-zyc-semeval-2026-task-5/