Elamur Research⌕
☰
home / papers / 2026 / Creating a Hybrid Rule and Neural Network Based Semantic Tagger Using Silver Standard Data: The PyMUSAS Framework for Multilingual Semantic Annotation

Creating a Hybrid Rule and Neural Network Based Semantic Tagger Using Silver Standard Data: The PyMUSAS Framework for Multilingual Semantic Annotation

Summary

The USAS semantic framework uses a coarse inventory of 232 category labels. Its tools have mostly been rule-based taggers driven by language-specific lexicons, and nobody had openly tested how well they assign tags in context across several languages. Hand-annotated USAS training data also did not exist. To fill this gap, the authors built an English silver-standard corpus of over 5 million tokens. They ran the original C implementation of the English USAS tagger over high-quality Wikipedia articles from MOSAICo Core. They then fine-tuned English and multilingual pre-trained language models that match tokens to tag glosses, training them with negative tags drawn from three frequency-weighted distributions. The neural taggers were compared with PyMUSAS rule-based taggers and with a hybrid of the two. The comparison used a new five-language benchmark (English, Finnish, Welsh, Irish, Chinese) that includes a newly annotated Chinese corpus. The work produces the first USAS-specific neural taggers for English, Irish, Finnish and Chinese. It also shows that a neural component can strengthen the rule-based system. The provided excerpt does not include the numeric scores. All models, data and code are openly released, which supports multilingual and lower-resource semantic annotation.

Key points

  • The USAS tagset has 21 major discourse fields and 232 category labels in up to three hierarchical levels. WordNet 3.0 has about 117.7k senses and BabelNet v5.3 about 22.9M. Evaluation ignores affix symbols and MWEs, and the neural models predict single labels only, with no dual or triple membership.
  • Silver training data came from MOSAICo Core English Wikipedia articles marked "good" or "featured". The texts were tokenized and POS-tagged with CLAWS and semantically tagged by the C-version tagger. Punctuation tags and Z99 (unmatched) tags were removed.
  • For each positive tag, three negative tags were sampled from three distributions: Original, Inverse frequency, and Log Inverse frequency. This counteracts the skewed label distribution and follows Blevins and Zettlemoyer (2020).
  • The evaluation data per language:
    • Finnish: a coffee-website corpus from Löfberg et al. (2003).
    • English: a post-edited machine translation of that Finnish corpus.
    • Welsh: the dataset from Ezeani et al. (2019).
    • Irish: data from Czerniak and Uí Dhonnchadha (2024), available on request.
    • Chinese: a new, openly released corpus, the first of its kind for USAS.
  • Released resources are hosted in three places:
    • GitHub UCREL/pymusas for the rule-based framework.
    • GitHub UCREL/experimental-wsd for the training code.
    • The Hugging Face collection ucrelnlp/usas-neural-taggers for the models, training data and evaluation sets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4ngkupgnrbgc

@inproceedings{moore2026creating,
  title = {{Creating a Hybrid Rule and Neural Network Based Semantic Tagger Using Silver Standard Data: The PyMUSAS Framework for Multilingual Semantic Annotation}},
  author = {Moore, Andrew and Rayson, Paul and Archer, Dawn and Czerniak, Tim and Knight, Dawn and Lal, Daisy Monika and Donnchadha, Gearóid Ó and Meachair, Mícheál J. Ó and Piao, Scott and Dhonnchadha, Elaine Uí and Vuorinen, Johanna and Yabo, Yan and Yang, Xiaobin},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4ngkupgnrbgc},
  url = {https://doi.org/10.63317/4ngkupgnrbgc}
}

References

37 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dawn Archer, Andrew Wilson, and Paul Rayson. 2002. Introduction to the USAS category system. find
  2. Edoardo Barba, Tommaso Pasini, and Roberto Navigli. 2021. ESC: Redesigning WSD with extractive sense comprehension. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4661–4672, Online. Association… find
  3. Pierpaolo Basile, Lucia Siciliani, Elio Musacchio, and Giovanni Semeraro. 2025. Exploring the word sense disambiguation capabilities of large language models. arXiv preprint arXiv:2503.08662. — [arXiv:2503.08662](https://arxiv.org/abs/2503.08662) find
  4. Terra Blevins and Luke Zettlemoyer. 2020. Moving down the long tail of word sense disambiguation with gloss informed bi-encoders. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1006–1017, Online. Association for Computational Linguistics. find
  5. Francis Bond and Ryan Foster. 2013. Linking and extending an open multilingual Wordnet. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1352–1362, Sofia, Bulgaria. Association for Computational Linguistics. find
  6. Ting-Yun Chang, Ta-Chung Chi, Shang-Chi Tsai, and Yun-Nung (Vivian) Chen. 2018. xSense: Learning sense-separated sparse representations and textual definitions for explainable word sense networks. ArXiv, abs/1809.03348. — [arXiv:1809.03348](https://arxiv.org/abs/1809.03348) find
  7. Simone Conia, Edoardo Barba, Abelardo Carlos Martinez Lorenzo, Pere-Lluís Huguet Cabot, Riccardo Orlando, Luigi Procopio, and Roberto Navigli. 2024. MOSAICo: a multilingual open-text semantically annotated interlinked corpus. In Proceedings of the 2024 Conference of the North American Chapter of the… find
  8. Tim Czerniak and Elaine Uí Dhonnchadha. 2024. Towards semantic tagging for Irish. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16643–16652, Torino, Italia. ELRA and ICCL. find
  9. Ignatius Ezeani, Scott Piao, Steven Neale, Paul Rayson, and Dawn Knight. 2019. Leveraging pre-trained embeddings for Welsh taggers. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 270–280, Florence, Italy. Association for Computational Linguistics. find
  10. Artyom Gadetsky, Ilya Yakubovskiy, and Dmitry Vetrov. 2018. Conditional generators of words definitions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 266– 271, Melbourne, Australia. Association for Computational Linguistic… find
  11. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations. find
  12. Helen Langone, Benjamin R. Haskell, and George A. Miller. 2004. Annotating WordNet. In Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004, pages 63–69, Boston, Massachusetts, USA. Association for Computational Linguistics. find
  13. Laura Löfberg, Dawn Archer, Scott Piao, Paul Rayson, Tony McEnery, Krista Varantola, and Jukka-Pekka Juntunen. 2003. Porting an English semantic tagger to the Finnish language. In Proceedings of the Corpus Linguistics 2003 conference., pages 457–464. find
  14. Laura Löfberg and Paul Rayson. 2019. Developing multilingual automatic semantic annotation systems. Cambridge University Press. find
  15. Marc Marone, Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, and Benjamin Van Durme. 2025. mmBERT: A modern multilingual encoder with annealed language learning. arXiv preprint arXiv:2509.06888. find
  16. Marco Maru, Simone Conia, Michele Bevilacqua, and Roberto Navigli. 2022. Nibbling at the hard core of Word Sense Disambiguation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4724–4737, Dublin, Ireland. Association for Compu… find
  17. Tom McArthur. 1981. Longman Lexicon of Contemporary English. Longman London. find
  18. Domenico Meconi, Simone Stirpe, Federico Martelli, Leonardo Lavalle, and Roberto Navigli. 2025. Do large language models understand word senses? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33885–33904, Suzhou, China. Association for Computational… find our card
  19. George A. Miller. 1995. WordNet: a lexical database for English. Commun. ACM, 38(11):39–41. find our card
  20. George A. Miller, Martin Chodorow, Shari Landes, Claudia Leacock, and Robert G. Thomas. 1994. Using a semantic concordance for sense identification. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994. find
  21. Roberto Navigli, Michele Bevilacqua, Simone Conia, Dario Montagnini, and Francesco Cecconi. 2021. Ten Years of BabelNet: A Survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4559–4567. International Joint Conferences on Artificial Intel… find
  22. Steven Neale, Kevin Donnelly, Gareth Watkins, and Dawn Knight. 2018. Leveraging lexical resources and constraint grammar for rule-based part-of-speech tagging in Welsh. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European… find
  23. Tommaso Pasini, Alessandro Raganato, and Roberto Navigli. 2021. Xl-wsd: An extra-large and cross-lingual evaluation framework for word sense disambiguation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15):13648–13656. find
  24. Scott Piao, Francesca Bianchi, Carmen Dayrell, Angela D’Egidio, and Paul Rayson. 2015. Development of the multilingual semantic annotation system. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages… find
  25. Scott Piao, Paul Rayson, Dawn Archer, Francesca Bianchi, Carmen Dayrell, Mahmoud El-Haj, Ricardo-María Jiménez, Dawn Knight, Michal Křen, Laura Löfberg, Rao Muhammad Adeel Nawab, Jawad Shafi, Phoey Lee Teh, and Olga Mudraya. 2016. Lexical coverage evaluation of large-scale multilingual semantic lexi… find
  26. Scott Piao, Paul Rayson, Dawn Knight, and Gareth Watkins. 2018. Towards a Welsh semantic annotation system. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  27. Scott S. L. Piao, Paul Rayson, Dawn Archer, Andrew Wilson, and Tony McEnery. 2003. Extracting multiword expressions with a semantic tagger. In Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment, pages 49–56, Sapporo, Japan. Association for Computationa… find
  28. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. 2017. Word sense disambiguation: A unified evaluation framework and empirical comparison. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 99–1… find our card
  29. Paul Rayson, Dawn E Archer, Scott L Piao, and Tony McEnery. 2004. The UCREL semantic analysis system. In Proceedings of the workshop on beyond named entity recognition semantic labelling for nlp tasks, in association with lrec-04, pages 7–12. European Language Resources Association. find
  30. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… find
  31. E. Uí Dhonnchadha and J. Van Genabith. 2006. A part-of-speech tagger for Irish using finite-state morphology and constraint grammar disambiguation. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), Genoa, Italy. European Language Resources Associati… find
  32. Loïc Vial, Benjamin Lecouteux, and Didier Schwab. 2018. UFSAC: Unification of sense annotated corpora and tools. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  33. John Vidler and Paul Rayson. 2024. UCREL - Hex; a shared, hybrid multiprocessor system. https://github.com/UCREL/hex. Accessed: 2024. ↗
  34. Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, and Benjamin Van Durme. 2025. Seq vs seq: An open suite of paired encoders and decoders. arXiv preprint arXiv:2507.11412. — [arXiv:2507.11412](https://arxiv.org/abs/2507.11412) find
  35. Guobiao Zhang, Wenpeng Lu, Xueping Peng, Shoujin Wang, Baoshuo Kan, and Rui Yu. 2022. Word sense disambiguation with knowledge-enhanced and local self-attention-based extractive sense comprehension. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4061–4070, Gy… find
  36. Zhi Zhong and Hwee Tou Ng. 2010. It makes sense: A wide-coverage word sense disambiguation system for free text. In Proceedings of the ACL 2010 System Demonstrations, pages 78–83, Uppsala, Sweden. Association for Computational Linguistics. 11. find
  37. Language Resource References Jialei Li, Mingchen Sun, and Jiajin Xu. 2022. ToRCH2019 Balanced Corpus of Modern Chinese. Center for Foreign Languages and Education, Beijing Foreign Studies University. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/moore-2026-creating-hybrid-rule-neural-network/