Elamur Research⌕
☰
home / papers / 2026 / Towards a Gold Standard for Adjectival Hypernymy: Enriching the Open English WordNet with a Hybrid Approach

Towards a Gold Standard for Adjectival Hypernymy: Enriching the Open English WordNet with a Hybrid Approach

Paper · 2026 · Lorenzo Augello, John P. McCrae and Marco Passarotti

Summary

Unlike nouns and verbs, adjectives in the Open English WordNet (OEWN) have no explicit hypernymy ("is-a") links. Adjectives are gradable, depend on context, and often span several semantic dimensions, so they are hard to place in a taxonomy. The only earlier gold set had just 302 English pairs, adapted from Dutch and Polish wordnets. That is too small for evaluating models or extending the OEWN. The authors first write linguistically grounded guidelines that define adjectival hypernymy and separate it from neighbouring relations. The same guidelines are used to prompt the models and to instruct human validators. Three LLMs receive a hyponym adjective together with its definition and propose candidate hypernyms. Humans then check every candidate. Candidates are grouped by how many models proposed them. The result is a validated resource of 3,836 adjective hyponym–hypernym pairs. Candidates that all three models agreed on were accepted 87% of the time. The dataset can be used to evaluate systems that detect hypernyms or recognise lexical entailment, and it provides material for adding adjectival hierarchies to the OEWN.

Key points

  • The final resource contains 3,836 human-validated adjective hyponym–hypernym pairs, up from a previous gold set of 302 pairs (Augello and McCrae, 2025). It is released on GitHub (lorenzoaugello/hypernymy).
  • Three LLMs generate candidates from a hyponym plus its definition. Outputs are labelled consensus (all models), majority, or singleton.
  • Consensus candidates reached 87% precision, so cross-model agreement works as a proxy for validity.
  • The resource covers a gap left by existing benchmarks. HyperLex covers only nouns and verbs. In ROOT9, adjectives make up only about 9% of hypernymy pairs, and ROOT9 has no theoretical guidelines.
  • Princeton WordNet and the OEWN link adjectives only via Similar To, See Also and antonymy. By contrast, the Polish, Dutch and German wordnets already organise adjectives hierarchically.

Interesting ideas

  • I1. One rulebook serves both the LLM prompt and the human annotation guide, so machine proposals and human judgments rest on identical criteria, which improves consistency and reproducibility.
  • I2. The number of independent models proposing an item is a cheap confidence signal: unanimous items are mostly correct and can be fast-tracked, while singletons get the closest human scrutiny.
  • I3. Feeding the gloss together with the word makes the model generate a hypernym for one sense, not for an ambiguous surface form like "hard" or "cold".
  • I4. Missing structure in a resource can be bootstrapped from sister-language wordnets that already encode it (Dutch, Polish, German adjective hierarchies) before scaling up with LLMs.
  • I5. Treating adjectival hypernymy as a directional entailment, not as mere similarity, addresses the known weakness of models that rely on distributional closeness.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova's LLM teacher tiers could be prompted with the same written criteria that reviewers use to accept silver labels, so every disagreement can be traced back to a specific rule.
  • I2 (learn): Silver data agreed by witness parsers could be split into consensus, majority and singleton bins, with a different review budget and acceptance policy for each bin.
  • I3 (world): When compiling the WordNet-derived ontology, adjective links could be keyed to the specific synset and definition, so the world model does not merge the "difficult" and "resistant" senses.
  • I4 (world): Adjective hierarchies borrowed through cross-lingual wordnets could give Mova property-level inference, e.g. "scarlet" implies "red", which in turn implies a colour attribute.
  • I5 (answer): Readers answering test questions could use explicit adjective-hypernym links to justify one-way inferences, such as "freezing" entailing "cold" but not the reverse, with each step explainable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksLexical semantics & word embeddings · 63 works

Cite

Cite as Lorenzo Augello, John P. McCrae and Marco Passarotti. 2026. Towards a Gold Standard for Adjectival Hypernymy: Enriching the Open English WordNet with a Hybrid Approach. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5eaqyq2rwc43

@inproceedings{augello2026gold,
  title = {{Towards a Gold Standard for Adjectival Hypernymy: Enriching the Open English WordNet with a Hybrid Approach}},
  author = {Lorenzo Augello, John P. McCrae and Marco Passarotti},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5eaqyq2rwc43},
  url = {https://doi.org/10.63317/5eaqyq2rwc43}
}

References

40 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lorenzo Augello and John Philip McCrae. 2025. Inferring adjective hypernyms with language models to increase the connectivity of Open English Wordnet. In Proceedings of the 5th Conference on Language, Data and Knowledge: The 5th OntoLex Workshop, pages 1–11, Naples, Italy. Unior Press. ↗
  2. Marco Baroni and Alessandro Lenci. 2011. How we BLESSed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 1–10, Edinburgh, UK. Association for Computational Linguistics.
  3. Georgeta Bordea, Els Lefever, and Paul Buitelaar. 2016. SemEval-2016 task 13: Taxonomy extraction evaluation (TExEval-2). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 1081–1091, San Diego, California. Association for Computational Linguistics. find
  4. Zied Bouraoui, José Camacho-Collados, and Steven Schockaert. 2019. Inducing relational knowledge from BERT. CoRR, abs/1911.12753. ↗
  5. Jose Camacho-Collados, Claudio Delli Bovi, Luis Espinosa-Anke, Sergio Oramas, Tommaso Pasini, Enrico Santus, Vered Shwartz, Roberto Navigli, and Horacio Saggion. 2018. SemEval-2018 task 9: Hypernym discovery. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 712–724, New Orleans, Louisiana. Association for Computational Linguistics. find
  6. D. A. Cruse. 1986. Lexical Semantics. Cambridge University Press, New York. find
  7. R. M. W. Dixon. 1982. Where have All the Adjectives Gone? De Gruyter Mouton, Berlin, New York. ↗
  8. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. The MIT Press. ↗
  9. Birgit Hamp and Helmut Feldweg. 1997. GermaNet - a lexical-semantic net for German. In Automatic Information Extraction and Building of Lexical Semantic Resources for NLP Applications.
  10. Yoshihiko Hayashi. 2025. Evaluating LLMs’ capability to identify lexical semantic equivalence: Probing with the word-in-context task. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6985–6998, Abu Dhabi, UAE. Association for Computational Linguistics. find
  11. Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14th Conference on Computational Linguistics - Volume 2, COLING ’92, page 539–545, USA. Association for Computational Linguistics.
  12. Frans Heyvaert. 2010. An outline for a semantic categorization of adjectives. In Proceedings of the 14th EURALEX International Congress, pages 1309–1318, Leeuwarden/Ljouwert, The Netherlands. Fryske Akademy. find
  13. Felix Hill, Roi Reichart, and Anna Korhonen. 2015. SimLex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 41(4):665–695. ↗
  14. Franz Hundsnurscher and Jochen Splett. 1982. Grundlegung Einer Semantischen Beschreibung der Adjektive des Deutschen, pages 16–47. VS Verlag für Sozialwissenschaften, Wiesbaden. find
  15. Omer Levy, Steffen Remus, Chris Biemann, and Ido Dagan. 2015. Do supervised distributional methods really learn lexical inference relations? In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 970–976, Denver, Colorado. Association for Computational Linguistics.
  16. John Lyons. 1977. Semantics. Cambridge University Press. find
  17. Marek Maziarz, Maciej Piasecki, Ewa Rudnicka, Stan Szpakowicz, and Paweł Kędzia. 2016. plWordNet 3.0 – a comprehensive lexical-semantic resource. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2259–2268, Osaka, Japan. find
  18. The COLING 2016 Organizing Committee. find
  19. John P. McCrae, Francesca Quattri, Christina Unger, and Philipp Cimiano. 2014. Modelling the semantics of adjectives in the ontology-lexicon interface. In Proceedings of the 4th Workshop on Cognitive Aspects of the Lexicon (CogALex), pages 198–209, Dublin, Ireland. Association for Computational Linguistics and Dublin City University. ↗
  20. Viktor Moskvoretskii, Alexander Panchenko, and Irina Nikishina. 2024. Are large language models good at lexical semantics? a case of taxonomy learning. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1498–1510, Torino, Italia. ELRA and ICCL. find
  21. M. Lynne Murphy. 2003. Semantic Relations and the Lexicon: Antonymy, Synonymy and other Paradigms. Cambridge University Press. find
  22. Roberto Navigli, Paola Velardi, and Juana Maria Ruiz-Martínez. 2010. An annotated dataset for extracting definitions and hypernyms from the web. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA). find
  23. Marten Postma, Emiel van Miltenburg, Roxane Segers, Anneleen Schoen, and Piek Vossen. 2016. Open Dutch WordNet. In Proceedings of the 8th Global WordNet Conference (GWC), pages 302–310, Bucharest, Romania. Global Wordnet Association. ↗
  24. Victor Raskin and Sergei Nirenburg. 1995. Lexical semantics of adjectives a microtheory of adjectival meaning. find
  25. Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. 2020. On the systematicity of probing contextualized word representations: The case of hypernymy in BERT. In Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics, pages 88–102, Barcelona, Spain (Online). Association for Computational Linguistics.
  26. Enrico Santus, Alessandro Lenci, Tin-Shing Chiu, Qin Lu, and Chu-Ren Huang. 2016. Nine features in a random forest to learn taxonomical semantic relations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4557–4564, Portorož, Slovenia. European Language Resources Association (ELRA). find
  27. Enrico Santus, Frances Yung, Alessandro Lenci, and Chu-Ren Huang. 2015. EVALution 1.0: an evolving semantic dataset for training and evaluation of distributional semantic models. In Proceedings of the 4th Workshop on Linked Data in Linguistics: Resources and Applications, pages 64–69, Beijing, China. Association for Computational Linguistics. find
  28. Silke Scheible and Sabine Schulte im Walde. 2014. A database of paradigmatic semantic relation pairs for German nouns, verbs, and adjectives. In Proceedings of Workshop on Lexical and Grammatical Resources for Language Processing, pages 111–119, Dublin, Ireland. Association for Computational Linguistics and Dublin City University. find
  29. Martin Schmitt and Hinrich Schütze. 2021. Language models for lexical inference in context. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1267– 1280, Online. Association for Computational Linguistics. find
  30. Yulia Tsvetkov, Nathan Schneider, Dirk Hovy, Archna Bhatia, Manaal Faruqui, and Chris Dyer. 2014. Augmenting English adjective senses with supersenses. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 4359–4365, Reykjavik, Iceland. European Language Resources Association (ELRA).
  31. Asahi Ushio, Luis Espinosa Anke, Steven Schockaert, and Jose Camacho-Collados. 2021. BERT is to NLP what AlexNet is to CV: Can pre-trained language models identify analogies? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3609–3624, Online. Association for Computational Linguistics. ↗
  32. Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. 2017. HyperLex: A large-scale evaluation of graded lexical entailment. Computational Linguistics, 43(4):781–835. ↗
  33. Ivan Vulić and Nikola Mrkšić. 2018. Specialising word vectors for lexical entailment. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1134–1145, New Orleans, Louisiana. Association for Computational Linguistics. 11.
  34. Language Resource References BNC Consortium. 2007. find
  35. British National Corpus 1994. Literary and Linguistic Data Service. find
  36. Fellbaum, Christiane. 1998. WordNet: An Electronic Lexical Database. The MIT Press. find
  37. Kilgarriff, Adam. 1997. Putting frequencies in the dictionary. ↗
  38. McCrae, John P. and Rademaker, Alexandre and Bond, Francis and Rudnicka, Ewa and Fellbaum, Christiane. 2019. find
  39. English WordNet 2019 – An Open-Source WordNet for English. Global Wordnet Association.
  40. Miller, George A. 1995. WordNet: a lexical database for English. Association for Computing Machinery.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/augello-2026-gold-standard-adjectival-hypernymy-enriching/