Elamur Research⌕
☰
home / papers / 2010 / Learning Word-Class Lattices for Definition and Hypernym Extraction

Learning Word-Class Lattices for Definition and Hypernym Extraction

Summary

The paper tackles automatic definition extraction: finding sentences that formally define a term, and pulling out the term's hypernym (genus) from them. Earlier approaches relied on hand-made or semi-learned lexico-syntactic patterns. These have poor recall and precision, because definitions vary widely in form and the common "X is a Y" construction is often not definitional. The authors introduce Word-Class Lattices (WCLs), an extension of word lattices. Sentences are first abstracted: frequent words stay as they are and all other words become their POS tags. Training definitions from Wikipedia are then grouped by wildcard "star patterns," and each group is merged into a lattice. Training sentences are annotated with definition fields (DF, VF, GF, REST), so one matching step both flags a definition and locates its hypernym. WCLs are evaluated on Wikipedia and on the ukWaC Web corpus against star patterns, the bigram soft-pattern model of Cui et al. (2007), and Hearst's patterns. On noisy Web text they achieve far higher precision than these alternatives. The method also needs essentially no parameter tuning and works across domains.

Key points

  • Definitions are split into four fields: DEFINIENDUM (DF), DEFINITOR (VF), DEFINIENS/genus (GF) and REST (RF). Text is POS-tagged with TreeTagger, and the defined term is replaced by a ⟨TARGET⟩ token.
  • On Wikipedia, WCL-1 and WCL-3 reach about 99% precision, versus 86% for star patterns. Star patterns and WCL-3 tie for the best F1 at 75%. Over 1,000 lattices were learned in total, including 381 DF, 252 VF and 395 GF lattices for WCL-3.
  • On ukWaC, recall was estimated from 50,000 manually checked sentences containing 99 definitions. WCL-3 scores 94.87% precision and 56.57% recall. Star patterns reach about 44% precision and 63% recall, and bigrams 46.60% precision and 45.45% recall.
  • On ukWaC hypernym extraction, WCL-3 gets 89.27% exact-match precision (383 hypernyms) and 96.27% substring precision. Hearst's patterns get 65.26% (62 hypernyms) and 88.42%.
  • On Wikipedia, exact-hypernym precision is only about 41–43%, while substring precision is about 77–79%. The only free parameter is the frequency threshold θ, which was not tuned. Datasets are released at lcl.uniroma1.it/wcl.

Interesting ideas

  • I1. Frequency-based abstraction to word classes. Keeping frequent function words literally and collapsing rare words to POS tags gives patterns that are specific enough to be precise yet general enough to transfer. This is a cheap trick that could suit pattern learning for low-resource languages, needing only a tagger.
  • I2. Cluster first, then generalize. Wildcard star patterns first group similar sentences, and only then is each group compressed into a lattice. This avoids aligning wildly different sentences, and the staged design could carry over to template induction for other relation types.
  • I3. Field-annotated training enables joint extraction. Labelling the parts of a definition in training means a match yields both the classification decision and the hypernym span. Annotating structure once and getting several tasks from it is a good economy for small annotation budgets.
  • I4. Cross-corpus evaluation to expose dataset bias. Testing on ukWaC showed that methods tuned on Wikipedia, where most "is a" sentences really are definitions, fall apart on Web text. Estimating recall from a hand-checked sample of a large corpus is a practical way to evaluate when full annotation is unaffordable.
  • I5. Reporting both exact and substring matches. Hypernym choice is often ambiguous, so scoring both exact and partial (substring) matches gives a fairer picture of span-extraction quality. The same approach is useful for any span task with fuzzy boundaries.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Lexical semantics & word embeddings

 Institutions

🎓Sapienza University of RomeRome · Italy29 works · 6,807 citations

Cite

Cite as Roberto Navigli and Paola Velardi. 2010. Learning Word-Class Lattices for Definition and Hypernym Extraction. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{navigli2010learning,
  title = {{Learning Word-Class Lattices for Definition and Hypernym Extraction}},
  author = {Roberto Navigli and Paola Velardi},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1134/}
}

References

43 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Ansa Olatz, Xabier Arregi, Xabier Artola, Arantza Daz de Ilarraza Snchez, Mikel Lersundi, David Martnez, Kepa Sarasola, and Ruben Urizar. 2000. Extraction of semantic relations from a basque monolingual dictionary using constraint grammar. In Proceedings of Euralex. find
  2. Claudia Borg, Mike Rosner, and Gordon Pace. 2009. Evolutionary algorithms for definition extraction. In Proceedings of the 1st Workshop on Definition Extraction 2009 (wDE’09). find
  3. William M. Campbell, M. F. Richardson, and D. A. Reynolds. 2007. Language recognition with word lattices and support vector machines. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2007), pages 989–992, Honolulu, HI. find
  4. Sharon A. Caraballo. 1999. Automatic construction of a hypernym-labeled noun hierarchy from text. In Proceedings of the 37 th Annual Meeting of the Association for Computational Linguistics (ACL), pages 120–126, Maryland, USA. our card
  5. Claudio Carpineto and Giovanni Romano. 2005. Using concept lattices for text retrieval and mining. In B. Ganter, G. Stumme, and R. Wille, editors, Formal Concept Analysis, pages 161–179. find
  6. Christopher Collins, Bob Carpenter, and Gerald Penn. 2004. Head-driven parsing for word lattices. In Proceedings of the 42nd Meeting of the Association for Of course, it would need some additional work if applied to languages other than English. However, the approach does not need to be adapted to t… find
  7. Computational Linguistics (ACL’04), Main Volume, pages 231–238, Barcelona, Spain, July. find
  8. Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. 1990. Introduction to algorithms. the MIT Electrical Engineering and Computer Science Series. MIT Press, Cambridge, MA. find
  9. Hang Cui, Min-Yen Kan, and Tat-Seng Chua. 2007. Soft pattern matching models for definitional question answering. ACM Transactions on Information Systems, 25(2):8. find
  10. Łukasz Degórski, Michał Marcinczuk, and Adam Przepiórkowski. 2008. Definition extraction using a sequential combination of baseline grammars and machine learning classifiers. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. find
  11. William Dolan, Lucy Vanderwende, and Stephen D. Richardson. 1993. Automatically deriving structured knowledge bases from on-line dictionaries. In Proceedings of the First Conference of the Pacific Association for Computational Linguistics, pages 5– 14. find
  12. Christopher Dyer, Smaranda Muresan, and Philip Resnik. 2008. Generalizing word lattice translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2008), pages 1012–1020, Columbus, Ohio, USA. our card
  13. Christopher Dyer. 2009. Using a maximum entropy model to build segmentation lattices for mt. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL 2009), pages 406–414, Boulder, Colorado, US… find
  14. Ismail Fahmi and Gosse Bouma. 2006. Learning to identify definitions using syntactic features. In Proceedings of the EACL 2006 workshop on Learning Structured Information in Natural Language Applications, pages 64–71, Trento, Italy. find
  15. Adriano Ferraresi, Eros Zanchetta, Marco Baroni, and Silvia Bernardini. 2008. Introducing and evaluating ukwac, a very large Web-derived corpus of english. In Proceedings of the 4th Web as Corpus Workshop (WAC-4), Marrakech, Morocco. find
  16. Aldo Gangemi, Roberto Navigli, and Paola Velardi. 2003. The OntoWordNet project: Extension and axiomatization of conceptual relations in WordNet. In Proceedings of the International Conference on Ontologies, Databases and Applications of SEmantics (ODBASE 2003), pages 820–838, Catania, Italy. find
  17. Rosa Del Gaudio and António Branco. 2007. Automatic extraction of definitions in portuguese: A rule-based approach. In Proceedings of the TeMa Workshop. find
  18. Marti Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14 th International Conference on Computational Linguistics (COLING), pages 539–545, Nantes, France. find
  19. Eduard Hovy, Andrew Philpot, Judith Klavans, Ulrich Germann, and Peter T. Davis. 2003. Extending metadata definitions by automatically extracting and organizing glossary definitions. In Proceedings of the 2003 Annual National Conference on Digital Government Research, pages 1–6. Digital Government S… find
  20. Adrian Iftene, Diana Trandabă, and Ionut Pistol. 2007. Natural language processing and knowledge representation for elearning environments. In Proc. of Applications for Romanian. Proceedings of RANLP workshop, pages 19–25. find
  21. Wenbin Jiang, Haitao Mi, and Qun Liu. 2008. Word lattice reranking for chineseword segmentation and part-of-speech tagging. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008), pages 385–392, Manchester, UK. find
  22. Judith Klavans and Smaranda Muresan. 2001. Evaluation of the DEFINDER system for fully automatic glossary construction. In Proc. of the American Medical Informatics Association (AMIA) Symposium. find
  23. Michael Tully Klein. 2008. Understanding English with Lattice-Learning, Master thesis. MIT, Cambridge, MA, USA. find
  24. Lambert Mathias and William Byrne. 2006. Statistical phrase-based speech translation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2006), Toulouse, France. find
  25. George A. Miller, R.T. Beckwith, Christiane D. Fellbaum, D. Gross, and K. Miller. 1990. WordNet: an online lexical database. International Journal of Lexicography, 3(4):235–244. find
  26. Roberto Navigli and Paola Velardi. 2006. Ontology enrichment through automatic semantic annotation of on-line glossaries. In Proceedings of the 15th International Conference on Knowledge Engineering and Knowledge Management (EKAW 2006), pages 126–140, Podebrady, Czech Republic. find
  27. Roberto Navigli, Paola Velardi, and Juana Marı́a Ruiz-Martı́nez. 2010. An annotated dataset for extracting definitions and hypernyms from the Web. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010), Valletta, Malta. find
  28. Roberto Navigli. 2009a. Using cycles and quasi-cycles to disambiguate dictionary glosses. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2009), pages 594–602, Athens, Greece. find
  29. Roberto Navigli. 2009b. Word Sense Disambiguation: A survey. ACM Computing Surveys, 41(2):1–69. find
  30. Michael P. Oakes. 2005. Using hearst’s rules for the automatic acquisition of hyponyms for mining a pharmaceutical corpus. In Proceedings of the Workshop Text Mining Research. find
  31. Adam Przepiórkowski, Lukasz Degórski, Beata Wójtowicz, Miroslav Spousta, Vladislav Kuboň, Kiril Simov, Petya Osenova, and Lothar Lemnitzer. 2007. Towards the automatic extraction of definitions in slavic. In Proceedings of the Workshop on Balto-Slavonic Natural Language Processing (in ACL ’07),… find
  32. Alan Ritter, Stephen Soderland, and Oren Etzioni. 2009. What is this, anyway: Automatic hypernym discovery. In Proceedings of the 2009 AAAI Spring Symposium on Learning by Reading and Learning to Read, pages 88–93. find
  33. Horacio Saggion. 2004. Identifying denitions in text collections for question answering. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal. find
  34. Antonio Sanfilippo and Victor Poznański. 1992. The acquisition of lexical knowledge from combined machine-readable dictionary sources. In Proceedings of the third Conference on Applied Natural Language Processing, pages 80–87. find
  35. Helmut Schmid. 1995. Improvements in part-of-speech tagging with an application to german. In Proceedings of the ACL SIGDAT-Workshop, pages 47–50. find
  36. Josh Schroeder, Trevor Cohn, and Philipp Koehn. 2009. Word lattices for multi-source translation. In Proceedings of the European Chapter of the Association for Computation Linguistics (EACL 2009), pages 719–727, Athens, Greece. find
  37. Rion Snow, Dan Jurafsky, and Andrew Y. Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In Proceedings of Advances in Neural Information Processing Systems, pages 1297–1304. find
  38. Angelika Storrer and Sandra Wellinghoff. 2006. Automated detection and annotation of term definitions in german text corpora. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), Genova, Italy. find
  39. Paola Velardi, Roberto Navigli, and Pierluigi D’Amadio. 2008. Mining the Web to create specialized glossaries. IEEE Intelligent Systems, 23(5):18–25. find
  40. Eline Westerhout and Paola Monachesi. 2007. Extraction of dutch definitory contexts for eLearning purposes. In Proceedings of CLIN. find
  41. Eline Westerhout. 2009. Definition extraction using linguistic and structural features. In Proceedings of the RANLP 2009 Workshop on Definition Extraction, pages 61–67. find
  42. Chunxia Zhang and Peng Jiang. 2009. Automatic extraction of definitions. In Proceedings of 2nd IEEE International Conference on Computer Science and Information Technology, pages 364–368. find
  43. Zhao-man Zhong, Zong-tian Liu, and Yan Guan. 2008. Precise information extraction from text based on two-level concept lattice. In Proceedings of the 2008 International Symposiums on Information Processing (ISIP ’08), pages 275–279, Washington, DC, USA. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/navigli-2010-learning-word-class-lattices-definition/