Elamur Research⌕
☰
home / papers / 2016 / A Large DataBase of Hypernymy Relations Extracted from the Web.

A Large DataBase of Hypernymy Relations Extracted from the Web.

Paper · 2016 · Julian Seitner, Christian Bizer, Kai Eckert et al.

Summary

Many NLP and data-integration tasks depend on knowing what type an entity belongs to. Popular knowledge bases cover well-known cities and celebrities but miss the long tail of obscure entities. To address this gap, the authors built the WebIsADb, an openly available collection of over 400 million hypernymy ("isa") tuples harvested from the CommonCrawl web corpus. They used the WebDataCommons framework on Amazon Web Services to apply Hearst-style lexico-syntactic patterns to billions of pages. Each extracted pair is stored as a rich tuple containing:

  • the hyponym and hypernym head nouns, with their pre- and post-modifiers,
  • the patterns that matched,
  • the source sentences and pay-level domains,
  • frequency and spread counts.

The tuples are indexed in MongoDB and can be queried through a Java API or downloaded as TSV files. The resource is far larger than Probase, the closest prior effort, which is also not freely available. It has already been used to reach competitive results in a SemEval taxonomy induction task. This makes it a practical foundation for taxonomy learning and other text-understanding applications.

Key points

  • The source corpus is CommonCrawl: more than 2.1 billion pages in over 38,000 WARC files (168 TB packed). Extraction took about 2,200 computing hours, run on 100 parallel servers in under a day.
  • The final database holds over 400 million tuples. By comparison, Probase extracted roughly 25 million pairs from 1.68 billion Bing pages.
  • The most productive pattern by far is "NP_t is a NP_h" (about 242 million matches). It is followed by "NP_h including NP_t" (~81M) and "NP_h such as NP_t" (~70M).
  • The pay-level domains contributing the most relations are blogspot.com (~32M), google.com (~31M) and wordpress.com (~11M).
  • Everything is released at webdatacommons.org/isadb/: a MongoDB dump, TSV tables, the Java API (class TuplesDb) and the extraction source code.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of StuttgartStuttgart · Germany95 works · 46,710 citations🎓University of MannheimMannheim · Germany31 works · 8,538 citations🎓Stuttgart Media UniversityStuttgart · Germany3 works · 148 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Lexical semantics & word embeddings · 69 works

Cite

Cite as Julian Seitner, Christian Bizer, Kai Eckert, Stefano Faralli, Robert Meusel, Heiko Paulheim and Simone Paolo Ponzetto. 2016. A Large DataBase of Hypernymy Relations Extracted from the Web. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{seitner2016large,
  title = {{A Large DataBase of Hypernymy Relations Extracted from the Web.}},
  author = {Julian Seitner, Christian Bizer, Kai Eckert, Stefano Faralli, Robert Meusel, Heiko Paulheim and Simone Paolo Ponzetto},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/2aqojnt6nje4},
  url = {https://doi.org/10.63317/2aqojnt6nje4}
}

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seitner-2016-large-database-hypernymy-relations-extracted/