Types of Common-Sense Knowledge Needed for Recognizing Textual Entailment

Summary

This short paper asks which kinds of world knowledge, beyond linguistic cues and lexical relations, people actually rely on when deciding Recognizing Textual Entailment (RTE) pairs. The authors started from the 600 Text–Hypothesis pairs of the RTE-5 corpus. They dropped UNKNOWN items and any pair that could be settled mainly through syntax, coreference, synonymy or hypernymy, which left 108 pairs. For each remaining pair, one author wrote a step-by-step informal proof. This yielded 221 pieces of background knowledge, which were grouped into 20 categories: form-based types such as preconditions and functionality, content-based types such as geography and arithmetic, and two miscellaneous types. Five non-expert annotators then labeled every statement. Overall agreement was solid, and the categories covered nearly all of the knowledge used. Only about 40% falls into types that existing resources and extraction systems already target. Definitions, arithmetic and accountability have been largely neglected. The study therefore points knowledge-base builders toward under-collected types of knowledge.

Key points

Data: 108 of 600 RTE-5 pairs were kept, yielding 221 world-knowledge statements. The 20 categories cover 215 of them (97%). Pairs, proofs and judgments were released publicly.
Agreement: Fleiss's κ is 0.678 across all categories. Arithmetic (0.968) and Geography (0.927) are most reliable. Prominent Relationship (0.145), Argument Types (0.180) and Simultaneous Conditions (0.203) scored poorly.
Frequency: Geography is the most common category (about 16.5% of statements), followed by Functionality (8.7%) and Definitions (7.8%).
Merging Functionality with Mutual Exclusivity raised κ to 0.784, suggesting a possible revision of the scheme. Merging Argument Types with Mutual Exclusivity lowered it to 0.410.
Familiar categories cover only about 40% of the knowledge. Among them are part–whole relations, membership, geographic facts, causal links, public entities and support/opposition, which map onto resources like WordNet and gazetteers. Preconditions and functionality have received only recent attention.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

41 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 36 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lobue2011types,
  title = {{Types of Common-Sense Knowledge Needed for Recognizing Textual Entailment}},
  author = {LoBue, Peter and Yates, Alexander},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-2057/}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

M. Banko, M. J. Cafarella, S. Soderland, M. Broadhead, and O. Etzioni. 2007. Open information extraction from the web. In IJCAI. find our card
Peter Clark, William R. Murray, John Thompson, Phil Harrison, Jerry Hobbs, and Christiane Fellbaum. 2007. On the role of lexical and world knowledge in rte3. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, RTE ’07, pages 54–59, Morristown, NJ, USA. Association for Computational Linguistics. find
I. Dagan, O. Glickman, and B. Magnini. 2006. The PAS-CAL Recognising Textual Entailment Challenge. Lecture Notes in Computer Science, 3944:177–190. find
Richard Feldman. 2002. Epistemology. Prentice Hall. find
Christiane Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. Bradford Books. find
R.V. Guha and D.B. Lenat. 1990. Cyc: a mid-term report. AI Magazine, 11(3). ↗
V. Vydiswaran M. Sammons and D. Roth. 2010. Ask not what textual entailment can do for you... In Proc. of the Annual Meeting of the Association of Computational Linguistics (ACL), Uppsala, Sweden, 7. Association for Computational Linguistics. find
Shachar Mirkin, Ido Dagan, and Eyal Shnarch. 2009. Evaluating the inferential utility of lexical-semantic resources. In EACL. find
Alan Ritter, Doug Downey, Stephen Soderland, and Oren Etzioni. 2008. It’s a contradiction — No, it’s not: A case study using functional relations. In Empirical Methods in Natural Language Processing. find
Dan Roth and Mark Sammons. 2007. Semantic and logical inference model for textual entailment. In Proceedings of ACL-WTEP Workshop. find
Avirup Sil, Fei Huang, and Alexander Yates. 2010. Extracting action and event semantics from web text. In AAAI Fall Symposium on Common-Sense Knowledge (CSK). find
D. G. Stork. 1999. The OpenMind Initiative. IEEE Expert Systems and Their Applications, 14(3):19–20. find
Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A core of semantic knowledge. In Proceedings of the 16th International Conference on the World Wide Web (WWW). find
Fei Wu and Daniel S. Weld. 2007. Automatically semantifying wikipedia. In Sixteenth Conference on Information and Knowledge Management (CIKM-07). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lobue-2011-types-common-sense-knowledge-needed/

📄 Types of Common-Sense Knowledge Needed for Recognizing Textual Entailment