Elamur Research⌕
☰
home / papers / 2021 / Exemplification Modeling: Can You Give Me an Example, Please?

Exemplification Modeling: Can You Give Me an Example, Please?

Summary

The authors introduce a new task called Exemplification Modeling (ExMod). A system is given one or more words together with their definitions, and it must compose a sentence in which each word is used precisely in the given sense. In other words, this is the inverse of Definition Modeling. For this task, ExMaker, an encoder-decoder model, is proposed. As input, it receives "lemma — definition" pairs marked with special tokens, and as output it produces a sentence with the target words marked. Lemmatization then recovers the positions at which these words occur. The model is trained on existing annotated corpora. For sense pairs, examples are selected in which both words occur close to each other and form a statistically significant collocation. Quality is assessed "in vivo": the generated sentences are added to the training data for Word Sense Disambiguation. With these data, the WSD model surpasses the previous state of the art as well as the result of training on gold corpora alone. Annotators rated the fluency and semantic adequacy of the sentences almost on par with manually built corpora. The practical value of the approach is that sense-tagged data can be generated on demand for the entire sense inventory.

Key points

  • ExMaker architecture. This is a sequence-to-sequence model. Lemmas in the input are wrapped in <s>/</s> tokens, and target spans in the sentence are wrapped in <t>/</t> markers. The model is trained with cross-entropy. It can generate examples for three senses at once, although only sets of one or two senses (k ∈ {1, 2}) were used during training.
  • Resources. The sense inventory is WordNet. Training data: SemCor (33K sentences, 200K instances), Princeton WordNet Gloss Corpus (WNG, 34,275 examples), and Oxford Dictionary Dataset (555,695 sentences with ODE senses). Sense collocations are taken from SyntagNet.
  • Example selection for sense pairs. Examples in which the two words are more than ξ tokens apart are discarded. The remaining ones are sampled with probability proportional to PPMI; the word frequencies for it are computed on English Wikipedia. Smoothing with α = 0.15 is additionally applied.
  • Evaluation. The evaluation consists of 5 WSD subtasks. Among them is a zero-shot setting: the model is trained on Oxford senses, and examples are generated for WordNet senses. Examples are generated for all lemma–definition pairs in WordNet. In this way, the authors partially address the problem that existing corpora cover less than 20% of WordNet senses.
  • Human evaluation and open code. The generated sentences are almost as fluent as those in manually built corpora, and their adequacy to the given sense is only slightly lower. The model, datasets, and code are released at github.com/SapienzaNLP/exmod.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Word sense disambiguation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.24963/ijcai.2021/520

@inproceedings{barba2021exemplification,
  title = {{Exemplification Modeling: Can You Give Me an Example, Please?}},
  author = {Barba, Edoardo and Procopio, Luigi and Lacerra, Caterina},
  booktitle = {IJCAI 2021},
  year = {2021},
  doi = {10.24963/ijcai.2021/520},
  url = {https://doi.org/10.24963/ijcai.2021/520}
}

References

32 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Edoardo Barba, Luigi Procopio, Niccolò Campolungo, Tommaso Pasini, and Roberto Navigli. MuLaN: Multilingual Label propagatioN for Word Sense Disambiguation. In Proc. of IJCAI, 2020. find
  2. Edoardo Barba, Tommaso Pasini, and Roberto Navigli. ESC: Redesigning WSD with extractive sense comprehension. In Proc. of NAACL, pages 4661–4672, 2021. find
  3. Michele Bevilacqua and Roberto Navigli. Breaking through the 80% glass ceiling: Raising the state of the art in word sense disambiguation by incorporating knowledge graph information. In Proc. of ACL, pages 2854– 2864, 2020. find
  4. Michele Bevilacqua, Marco Maru, and Roberto Navigli. Generationary or: “how we went beyond word sense inventories and learned to gloss”. In Proc. of EMNLP, pages 7207–7221, 2020. find our card
  5. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, and Roberto Navigli. Recent trends in Word Sense Disambiguation: A survey. In Proc. of IJCAI, 2021. find
  6. Terra Blevins and Luke Zettlemoyer. Moving down the long tail of word sense disambiguation with gloss informed bi-encoders. In Proc. of ACL, pages 1006– 1017, 2020. find
  7. Ting-Yun Chang, Ta-Chung Chi, Shang-Chi Tsai, and Yun-Nung Chen. xSense: Learning sense-separated sparse representations and textual definitions for explainable word sense networks. arXiv preprint arXiv:1809.03348, 2018. — [arXiv:1809.03348](https://arxiv.org/abs/1809.03348) find
  8. Jacob Cohen. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46, 1960. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL, pages 4171–4186, 2019. find
  10. Artyom Gadetsky, Ilya Yakubovskiy, and Dmitry Vetrov. Conditional generators of words definitions. In Proc. of ACL, pages 266–271, 2018. find
  11. Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, and Masaru Kitsuregawa. Learning to describe unknown phrases with local and global contexts. In Proc. of NAACL, pages 3467–3476, 2019. find
  12. J Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data. biometrics, pages 159–174, 1977. find
  13. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proc. of ACL, pages 7871–7880, 2020. find our card
  14. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proc. of EMNLP, pages 2122–2132, 2016. find
  15. Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. In ICLR, 2020. find
  16. Marco Maru, Federico Scozzafava, Federico Martelli, and Roberto Navigli. SyntagNet: Challenging supervised word sense disambiguation with lexical-semantic combinations. In Proc. of EMNLP, pages 3534–3540, 2019. find
  17. Timothee Mickus, Denis Paperno, and Matthieu Constant. Mark my word: A sequence-to-sequence approach to definition modeling. In Proc. of DeepNLP, 2019. find
  18. George A. Miller, R.T. Beckwith, Christiane D. Fellbaum, D. Gross, and K. Miller. Introduction to WordNet: an online lexical database. International Journal of Lexicography, 3(4):235–244, 1990. find
  19. George A Miller, Claudia Leacock, Randee Tengi, and Ross T Bunker. A semantic concordance. In Proc. of Human Language Technology, pages 303–308, 1993. find
  20. Andrea Moro and Roberto Navigli. SemEval-2015 task 13: Multilingual all-words sense disambiguation and entity linking. In Proc. of SemEval, pages 288– 297, 2015. find
  21. Ian SP Nation. Learning vocabulary in another language. Cambridge University Press, 2001. find
  22. Roberto Navigli, David Jurgens, and Daniele Vannella. SemEval-2013 task 12: Multilingual word sense disambiguation. In Proc. of SemEval, pages 222–231, 2013. find
  23. Roberto Navigli. Word Sense Disambiguation: A survey. ACM computing surveys (CSUR), 41(2):1–69, 2009. find
  24. Yoshiki Niwa and Yoshihiko Nitta. Co-occurrence vectors from corpora vs. distance vectors from dictionaries. In Proc. of COLING, 1994. find
  25. Thanapon Noraset, Chen Liang, Larry Birnbaum, and Doug Downey. Definition modeling: Learning to define word embeddings in natural language. In Proc. of AAAI, 2017. find
  26. Martha Palmer, Christiane Fellbaum, Scott Cotton, Lauren Delfs, and Hoa Trang Dang. English tasks: All-words and verb lexical sample. In Proc. of Senseval-2, pages 21–24, 2001. find
  27. Sameer Pradhan, Edward Loper, Dmitriy Dligach, and Martha Palmer. SemEval-2007 task-17: English lexical sample, SRL and all words. In Proc. of SemEval, pages 87–92, 2007. find
  28. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. Word Sense Disambiguation: A unified evaluation framework and empirical comparison. In Proc. of EACL, pages 99–110, 2017. find our card
  29. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. Just “OneSeC” for producing multilingual sense-annotated data. In Proc. of ACL, pages 699–709, 2019. find
  30. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. With more contexts comes better performance: Contextualized sense embeddings for all-round word sense disambiguation. In Proc. of EMNLP, pages 3528–3539, 2020. find
  31. Benjamin Snyder and Martha Palmer. The English all-words task. In Proc. of Senseval-3, pages 41– 43, 2004. find
  32. Kaveh Taghipour and Hwee Tou Ng. One million sense-tagged instances for word sense disambiguation and induction. In Proc. of CoNLL, pages 338–344, 2015. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/barba-2021-exemplification-modeling/