Elamur Research⌕
☰
home / papers / 2024 / Coarse-Grained Sense Inventories Based on Semantic Matching between English Dictionaries

Coarse-Grained Sense Inventories Based on Semantic Matching between English Dictionaries

Paper · 2024 · Masato Kikuchi, Masatsugu Ono, Toshioki Soga et al.

Summary

WordNet splits word meanings so finely that even people often cannot tell its senses apart. This hurts word sense disambiguation (WSD) and makes WordNet hard to use in language teaching. The authors build coarser inventories automatically by aligning WordNet definitions with those of two Cambridge dictionaries: the Cambridge Learner's Dictionary (CLD) and the Cambridge English Dictionary (CED). They take 15,885 words from Japanese English-learning vocabulary lists. gpt-4o rates every same-PoS pair of definitions on a seven-step similarity scale. Pairs judged identical or nearly identical count as matches, and the WordNet senses that map to one Cambridge sense are merged into a group. The result is 3,222 (CLD) and 9,457 (CED) groups, each tagged with a CEFR level. The authors compare their groups with the manually built, class-based CSI and report that their groups merge closely related senses more coherently. Re-scoring the ConSeC WSD model with these groups suggests its realistic accuracy is roughly 85–86%. The method needs no sense-tagged corpus, so it reaches rare senses and is cheap to extend.

Key points

  • Matching used gpt-4o-2024-05-13 at temperature 0, with a "professional linguist" system role. The prompt offered a 7-point scale, and only answers 1–2 counted as matches.
  • The CLD–WordNet inventory has 3,222 sense groups and the CED–WordNet inventory has 9,457. Both are released on Zenodo (doi 10.5281/zenodo.13706831).
  • The baseline CSI was built by manually clustering Roget Thesaurus categories, and annotators then labelled WordNet synsets. It covers about 83K synsets (70.4%), but it is coarse and hard to scale.
  • The WSD check used the ALL dataset (Senseval/SemEval). Of its 7,253 targets, 6,605 were in the vocabulary lists. On that subset, ConSeC's fine-grained score is 82.1%, versus 83.2% on the full set.
  • Counting near-miss predictions inside the same group as correct raises ConSeC to about 85% (CLD) and 86% (CED). CSI gives a much larger jump, which the authors suspect overstates performance.

Interesting ideas

  • I1. Ask the LLM for a graded similarity rating and set the acceptance threshold afterwards, instead of asking yes/no; this makes the strictness of matching a tunable knob without re-querying.
  • I2. Build sense groups purely from aligning definitions across two dictionaries; no tagged corpus is needed, so rare senses are handled as well as frequent ones.
  • I3. Score a fine-grained system leniently by accepting a wrong prediction if it falls in the gold sense's coarse group; this separates genuine errors from near-misses.
  • I4. Prune comparisons by requiring matching PoS before pairing definitions; this cuts cost and avoids nonsense alignments.
  • I5. Tag each coarse sense with a CEFR level, so senses carry a learner-difficulty priority.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): LLM teacher tiers could rate silver-data agreement on a graded scale, so the cutoff for accepting a parse or frame can be tuned against negative controls.
  • I2 (world): the WordNet-based ontology could gain a coarse sense layer, letting frames and entities attach to a stable concept when fine senses are indistinguishable in context.
  • I3 (learn): evaluation of sense choices in frames could report strict and group-lenient scores side by side, with an over-coarse inventory as a control that shows inflated gains.
  • I4 (tokens): lemma-to-sense lookups could be filtered by the tagger's UPOS first, so only same-PoS WordNet senses ever reach later stages.
  • I5 (answer): readers solving school-course tests could prefer lower-CEFR senses when interpreting simple texts and explain that choice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hokkaido UniversitySapporo · Japan4 works · 17 citations🎓Nagoya Institute of TechnologyNagoya · Japan3 works · 3 citations

Also on the byline: Muroran Institute of Technology · Hokkaido University of Science · Chitose Institute of Science and Technology · Hokkaido University of Education.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Masato Kikuchi, Masatsugu Ono, Toshioki Soga, Tetsu Tanabe and Tadachika Ozono. 2024. Coarse-Grained Sense Inventories Based on Semantic Matching between English Dictionaries. arXiv:2409.06386.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2409.06386 · arXiv 2409.06386

@misc{kikuchi2024coarse,
  title = {{Coarse-Grained Sense Inventories Based on Semantic Matching between English Dictionaries}},
  author = {Masato Kikuchi, Masatsugu Ono, Toshioki Soga, Tetsu Tanabe and Tadachika Ozono},
  year = {2024},
  doi = {10.48550/arxiv.2409.06386},
  eprint = {2409.06386},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2409.06386}
}

References

19 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C. Fellbaum, WordNet: An Electronic Lexical Database. The MIT Press, 1998. find
  2. C. Lacerra, M. Bevilacqua, T. Pasini, and R. Navigli, “CSI: A coarse sense inventory for 85% word sense disambiguation,” in Proceedings of the AAAI 2020, 2020, pp. 8123–8130. find
  3. R. Izquierdo, A. Suarez, and G. Rigau, “Word vs. classbased word sense disambiguation,” Journal of Artificial Intelligence Research, vol. 54, pp. 83–122, 2015. find
  4. M. Palmer, O. Babko-Malaya, and H. T. Dang, “Different sense granularities for different applications,” in Proceedings of the ScaNaLU 2004, 2004, pp. 49–56. find
  5. E. Hovy, M. Marcus, M. Palmer, L. Ramshaw, and R.Weischedel, “OntoNotes: The 90% solution,” in Proceedings of the NAACL 2006, 2006, pp. 57–60. find
  6. R. Navigli, “Meaningful clustering of senses helps boost word sense disambiguation performance,” in Proceedings of the COLING-ACL 2006, 2006, pp. 105–112.
  7. M. Palmer, H. T. Dang, and C. Fellbaum, “Making fine-grained and coarse-grained sense distinctions, both manually and automatically,” Natural Language Engineering, vol. 13, no. 2, pp. 137–163, 2007. find
  8. R. Snow, S. Prakash, D. Jurafsky, and A. Y. Ng, “Learning to merge word senses,” in Proceedings of the EMNLP-CoNLL 2007, 2007, pp. 1005–1014. find
  9. R. Izquierdo, A. Suarez, and G. Rigau, “Exploring the automatic selection of basic level concepts,” in Proceedings of the RANLP 2007, 2007, pp. 1–5. find
  10. L. Vial, B. Lecouteux, and D. Schwab, “Sense vocabulary compression through the semantic knowledge of wordnet for neural word sense disambiguation,” in Proceedings of the GWC 2019, 2019, pp. 108–117. find
  11. B. Magnini and G. Cavagliá, “Integrating subject field codes into WordNet,” in Proceedings of the LREC 2000, 2000, pp. 1413–1418.
  12. J. Camacho-Collados and R. Navigli, “BabelDomains: Large-scale domain labeling of lexical resources,” in Proceedings of the EACL 2017, 2017, pp. 223–228. find
  13. T. Aiba, “Are groupings for word sense definitions that boost inter-annotator agreement the same as groupings based on language learner ability?” Graduation Thesis, Hirosaki University, 2021. find
  14. D. Chandrasekaran and V. Mago, “Evolution of semantic similarity – A survey,” ACM Computing Survey, vol. 54, no. 2, 41, pp. 1–37, 2021. find
  15. D. W. Prakoso, A. Abdi, and C. Amrit, “Short text similarity measurement methods: A review,” Soft Computing, vol. 25, pp. 4699–4723, 2021. find
  16. W. Yao, X. Pan, L. Jin, J. Chen, D. Yu, and D. Yu, “Connect-the-Dots: Bridging semantics between words and definitions via aligning word sense inventories,” in Proceedings of the EMNLP 2021, 2021, pp. 7741– 7751. find
  17. G. K. Zipf, “The meaning-frequency relationship of words,” Journal of General Psychology, vol. 33, no. 2, pp. 251–256, 1945. find
  18. A. Raganato, J. Camacho-Collados, and R. Navigli, “Word sense disambiguation: A unified evaluation framework and empirical comparison,” in Proceedings of the EACL 2017, 2017, pp. 99–110.
  19. E. Barba, L. Procopio, and R. Navigli, “ConSeC: Word sense disambiguation as continuous sense comprehension,” in Proceedings of the EMNLP 2021, 2021, pp. 1492–1503.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kikuchi-2024-coarse-grained-sense-inventories-based/