Elamur Research⌕
☰
home / papers / 2015 / Recognizing Textual Entailment Using Probabilistic Inference

Recognizing Textual Entailment Using Probabilistic Inference

Paper · 2015 · Lei Sha, Sujian Li, Baobao Chang et al.

Summary

The paper addresses Recognizing Textual Entailment (RTE), which asks whether a hypothesis H follows from a text T. The authors see two problems with earlier work. Shallow overlap systems miss facts that are only implied. Converting sentences into strict logic loses information and makes proofs fragile.

Their pipeline has four steps:

  • Decompose. T and H are broken into simple discourse commitments, following Hickl (2008).
  • Extract triples. ReVerb turns each commitment into a predicate–argument triple, and every argument is replaced by a matching noun phrase.
  • Align to YAGO. Triples are matched to YAGO relations. When no direct match exists, paraphrases derived from DIRT are tried.
  • Infer. AMIE mines inference rules from YAGO, which become a Markov Logic Network with weights fitted to YAGO facts. Alchemy scores candidate new facts, and T is judged to entail H if the inferred facts cover H's predicates.

On the combined RTE-2 and RTE-3 data, the system reaches 85.16% accuracy, a small but statistically significant gain over the commitment-only baseline. The work shows that weighted, defeasible rules can recover implied facts without a full logical translation.

Key points

  • Evaluation covers PASCAL RTE-2 and RTE-3, 1600 T–H pairs in total. Accuracy is 85.16% against 84.93% for Hickl (2008), significant by Wilcoxon signed-rank test (p < 0.05). The strict-logic system of Tatu and Moldovan reaches only 71.59%.
  • YAGO2, with over 940K facts and about 470K entities, serves both for predicate alignment and for rule mining. A single AMIE run yielded roughly 1.8K inference rules.
  • Unmatched predicates fall back to their top-10 DIRT neighbours, learned from 1GB of randomly sampled Gigaword text. If none of these match either, the predicate gets no knowledge-base support.
  • For each pair, the MLN is built only from rules that mention at least one predicate occurring in that pair. Inference is run only when commitment-based paraphrasing fails to find entailment, making it a back-off step.
  • In the paper's example, the system recovers that Gracia is Elicia's mother from a sentence mentioning Hughes's wife and daughter, a case that requires reasoning.

Interesting ideas

  • I1. Breaking a complex sentence into several short standalone propositions before extracting relations makes facts hidden in relative clauses and appositions explicit and easy to match.
  • I2. Forcing every relation argument to coincide with a full noun-phrase span keeps extracted triples clean and joinable across sentences.
  • I3. Rules mined automatically from a knowledge base receive weights fitted to that base's own facts, so implied facts arrive with a confidence instead of a hard yes or no.
  • I4. Running the costly probabilistic reasoner only when a cheap aligner cannot decide keeps precision on easy cases and spends effort only on hard ones.
  • I5. When a relation name is missing from the knowledge base, retrying with a short list of distributional near-synonyms extends coverage without manual mapping.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could emit a separate frame for each relative clause, apposition and temporal modifier, so that "a doctor who lives in Austin" yields its own lives-in frame linked to the main event.
  • I2 (tree): after UD parsing, frame arguments could be normalized to the head's full nominal subtree, giving the world layer consistent entity keys.
  • I3 (world): kinship and location rules over the Wikidata-backed graph could add implied holdings or relations to the text world, each tagged with a weight and the rule that produced it.
  • I4 (answer): the readers could first try direct frame matching against the question and call graph inference only on failure, reporting which path produced the answer.
  • I5 (world): verbs with no ontology match could be retried through their nearest VerbNet-class or corpus-similar predicates before being marked as unsupported.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China100 works · 35,618 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 510 works

Cite

Cite as Lei Sha, Sujian Li, Baobao Chang, Zhifang Sui and Tingsong Jiang. 2015. Recognizing Textual Entailment Using Probabilistic Inference. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d15-1185

@inproceedings{sha2015recognizing,
  title = {{Recognizing Textual Entailment Using Probabilistic Inference}},
  author = {Lei Sha, Sujian Li, Baobao Chang, Zhifang Sui and Tingsong Jiang},
  booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1185},
  url = {https://doi.org/10.18653/v1/d15-1185}
}

References

21 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Islam Beltagy, Cuong Chau, Gemma Boleda, Dan Garrette, Katrin Erk, and Raymond Mooney. 2013. Deep semantics with probabilistic logical form. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics (* SEM-13). find
  2. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pages 177– 190. Springer. find
  3. Rodrigo de Salvo Braz, Roxana Girju, Vasin Punyakanok, Dan Roth, and Mark Sammons. 2006. An inference model for semantic entailment in natural language. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 261–286. Springer. find
  4. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1535–1545. Association for Computational Linguistics. find
  5. Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian Suchanek. 2013. Amie: association rule mining under incomplete evidence in ontological knowledge bases. In Proceedings of the 22nd international conference on World Wide Web, pages 413–422. International World Wide Web Conferences Steering Committee. find
  6. Aria D Haghighi, Andrew Y Ng, and Christopher D Manning. 2005. Robust textual inference via graph matching. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 387–394. Association for Computational Linguistics. find
  7. Andrew Hickl, John Williams, Jeremy Bensley, Kirk Roberts, Bryan Rink, and Ying Shi. 2006. Recognizing textual entailment with lccs groundhog system. In Proceedings of the Second PASCAL Challenges Workshop. find
  8. Andrew Hickl. 2008. Using discourse commitments to recognize textual entailment. In Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1, pages 337–344. Association for Computational Linguistics. find
  9. Valentin Jijkoun and Maarten de Rijke. 2005. Recognizing textual entailment using lexical similarity. In Proceedings of the PASCAL Challenges Workshop on Recognising Textual Entailment, pages 73–76. find
  10. Dekang Lin and Patrick Pantel. 2001. Dirt@ sbt@ discovery of inference rules from text. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 323–328. ACM. find
  11. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 41–48. Association for Computational Linguistics. find
  12. Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In 7th Biennial Conference on Innovative Data Systems Research. CIDR 2015.
  13. Prodromos Malakasiotis and Ion Androutsopoulos. 2007. Learning textual entailment using svms and string similarity measures. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 42–47. Association for Computational Linguistics. find
  14. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 1003–1011. Association for Computational Linguistics. find
  15. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine learning, 62(1-2):107–136. ↗
  16. Miguel Rios, Lucia Specia, Alexander Gelbukh, and Ruslan Mitkov. 2014. Statistical relational learning to recognise textual entailment. In Computational Linguistics and Intelligent Text Processing, pages 330–339. Springer. find
  17. Marta Tatu and Dan Moldovan. 2006. A logic-based semantic approach to recognizing textual entailment. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 819–826. Association for Computational Linguistics. find
  18. Lucy Vanderwende, Arul Menezes, and Rion Snow. 2006. Microsoft research at rte-2: Syntactic contributions in the entailment task: an implementation. In Proceedings of the Second PASCAL Challenges Workshop. find
  19. Stephen Wan, Mark Dras, Robert Dale, and Cécile Paris. 2006. Using dependency-based features to take the para-farce out of paraphrase. In Proceedings of the Australasian Language Technology Workshop, volume 2006. find
  20. Andreas Wotzlaw and Ravi Coote. 2013. A logic-based approach for recognizing textual entailment supported by ontological background knowledge. arXiv preprint arXiv:1310.4938. ↗
  21. Fabio Massimo Zanzotto and Alessandro Moschitti. 2006. Automatic learning of textual entailments with cross-pair similarities. In International conference on computational linguistics and the 44th Annual meeting of the Association for computational linguistics, volume 1, pages 401–408. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sha-2015-recognizing-textual-entailment-probabilistic-inference/