Elamur Research⌕
☰
home / papers / 2017 / Domain-Targeted, High Precision Knowledge Extraction

Domain-Targeted, High Precision Knowledge Extraction

Paper · 2017 · Bhavana Dalvi Mishra, Niket Tandon and Peter Clark

Summary

Science question-answering systems for school-level material need large collections of general (subject,predicate,object) facts. Existing KBs such as NELL, ConceptNet, WordNet, WebChild or ReVerb-15M fall short: they are noisy, centred on named entities, or limited to a handful of relations. The authors (Allen Institute for AI, TACL 2017) build an extraction pipeline aimed at a single domain, elementary science.

The pipeline works in several stages:

  • It starts from a 4th-grade vocabulary with one manually assigned sense per noun and a small inventory of entity types.
  • It retrieves Web sentences through Bing and runs Open IE over them.
  • Turkers label sample tuples, and a precision predictor filters the output.

A new algorithm, CASI, groups typed verb schemas that mean roughly the same thing and maps each group to one canonical predicate. It draws on both lexical resources and corpus evidence.

To judge coverage, the authors propose "comprehensiveness": the recall of facts from an independent science corpus while precision stays above 80%. The released Aristo Tuple KB scores 23% on this metric, well ahead of comparable resources. This shows that focused, reliable knowledge can be harvested with modest human supervision.

Key points

  • Domain inputs: about 6000 nouns, 2000 verbs, 2000 adjectives and 600 adverbs, plus a hand-built inventory of 45 entity types (e.g., animal, body part, measuring instrument).
  • Verbs are disambiguated through typed schemas such as (Animal, "have", BodyPart) rather than explicit verb senses.
  • Final KB: 340,163 triples at 80.6% precision, released as Aristo Tuple KB (allenai.org/data/aristo-tuple-kb).
  • Comprehensiveness rises from 20% (no canonicalization) to 20.9% with the AMIE* baseline and to 23.2% with CASI; the gain over the original KB is significant at 99% confidence.
  • From 15.8K input schemas, CASI yields 4.2K mapping rules and 2.5K canonical schemas, against 822 and 102 for AMIE*. Rule precision is 68% for CASI versus 59% for AMIE*.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States317 works · 106,633 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksBiochemistry, Genetics and Molecular Biology · 273 worksMolecular Biology · 270 works

Cite

Cite as Bhavana Dalvi Mishra, Niket Tandon and Peter Clark. 2017. Domain-Targeted, High Precision Knowledge Extraction. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00058

@article{mishra2017domain,
  title = {{Domain-Targeted, High Precision Knowledge Extraction}},
  author = {Bhavana Dalvi Mishra, Niket Tandon and Peter Clark},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2017},
  doi = {10.1162/tacl\_a\_00058},
  url = {https://doi.org/10.1162/tacl_a_00058}
}

References

33 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Auer, C. Bizer, J. Lehmann, G. Kobilarov, R. Cyganiak, and Z. Ives. 2007. DBpedia: A nucleus for a web of open data. In In ISWC/ASWC. find
  2. Michele Banko, Michael J. Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In IJCAI, volume 7, pages 2670–2676.
  3. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2011. Global learning of typed entailment rules. In ACL.
  4. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In SIGMOD. find
  5. Thorsten Brants and Alex Franz. 2006. Web 1T 5-gram version 1 LDC2006T13. find
  6. Philadelphia: Linguistic Data Consortium. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka Jr, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI, volume 5, page 3. find
  7. Bhavana Dalvi, Sumithra Bhakthavatsalam, Chris Clark, Peter Clark, Oren Etzioni, Anthony Fader, and Dirk Groeneveld. 2016. IKE - An Interactive Tool for Knowledge Extraction. In AKBC@NAACL-HLT. find
  8. Luciano Del Corro, Rainer Gemulla, and Gerhard Weikum. 2014. Werdy: Recognition and disambiguation of verbs and verb phrases with syntactic and semantic pruning. In 2014 Conference on Empirical Methods in Natural Language Processing, pages 374– 385. find
  9. ACL. Karthik Dinakar, Birago Jones, Catherine Havasi, Henry Lieberman, and Rosalind Picard. 2012. Common sense reasoning for detection, prevention, and mitigation of cyberbullying. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(3):18. find
  10. Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In KDD. find
  11. Alexander Faaborg and Henry Lieberman. 2006. A goal-oriented web browser. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 751–760. find
  12. ACM. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1535–1545. Association for Computational Linguistics. ReVerb-15M available at http://openi… find
  13. Christiane Fellbaum. 1998. WordNet. Wiley Online Library. Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2013. AMIE: association rule mining under incomplete evidence in ontological knowledge bases. In WWW. Luis Galárraga, Geremy Heitz, Kevin Murphy, and Fabian M. Suchane… find
  14. Jonathan Gordon and Lenhart K Schubert. 2010. Quantificational sharpening of commonsense knowledge. In AAAI Fall Symposium: Commonsense Knowledge. find
  15. Adam Grycner and Gerhard Weikum. 2014. Harpy: Hypernyms and alignment of relational paraphrases. In COLING. find
  16. Adam Grycner, Gerhard Weikum, Jay Pujara, James R. Foulds, and Lise Getoor. 2015. RELLY: Inferring hypernym relationships between relational phrases. In EMNLP. find
  17. Dekang Lin and Patrick Pantel. 2001. DIRT - discovery of inference rules from text. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 323– 328. find
  18. ACM. Hugo Liu and Push Singh. 2004. ConceptNet: a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211–226. find
  19. Mausam, Michael Schmitz, Stephen Soderland, Robert Bart, and Oren Etzioni. 2012. Open language learning for information extraction. In EMNLP. find
  20. Andrea Moro and Roberto Navigli. 2012. WiSeNet: building a wikipedia-based semantic network with on-tologized relations. In CIKM. find
  21. Patrick Pantel, Rahul Bhagat, Bonaventura Coppola, Timothy Chklovski, and Eduard H Hovy. 2007. ISP: Learning inferential selectional preferences. In HLT-NAACL, pages 564–571.
  22. Hannes Planatscher and Michael Schober. 2015. SCP solver. http://scpsolver.org. ↗
  23. Simon Razniewski, Fabian M Suchanek, and Werner Nutt. 2016. But what do we actually know? In Proc. AKBC’16. find
  24. Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In HLT-NAACL. find
  25. Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NIPS.
  26. Stephen Soderland, John Gilmer, Robert Bart, Oren Etzioni, and Daniel S. Weld. 2013. Open Information Extraction to KBP Relations in 3 Hours. In TAC. find
  27. Robert Speer and Catherine Havasi. 2013. Conceptnet 5: A large semantic network for relational knowledge. In The Peoples Web Meets NLP, pages 161–176. find
  28. Springer. Gabriel Stanovsky and Ido Dagan. 2016. Creating a large benchmark for open information extraction. In EMNLP. find
  29. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A Core of Semantic Knowledge. In WWW. find
  30. Niket Tandon, Gerard de Melo, Fabian Suchanek, and Gerhard Weikum. 2014. WebChild: Harvesting and Organizing Commonsense Knowledge from the Web. In WSDM. find
  31. Peter D. Turney, Yair Neuman, Dan Assaf, and Yohai Cohen. 2011. Literal and metaphorical sense identification through concrete and abstract context. In EMNLP.
  32. Travis Wolfe, Mark Dredze, James Mayfield, Paul Mc-Namee, Craig Harman, Timothy W. Finin, and Benjamin Van Durme. 2015. Interactive knowledge base population. CoRR, abs/1506.00301. — [arXiv:1506.00301](https://arxiv.org/abs/1506.00301) find
  33. Alexander Yates and Oren Etzioni. 2009. Unsupervised methods for determining object and relation synonyms on the web. Journal of Artificial Intelligence Research. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mishra-2017-domain-targeted-high-precision-knowledge/