Elamur Research⌕
☰
home / papers / 2017 / On-demand Injection of Lexical Knowledge for Recognising Textual Entailment

On-demand Injection of Lexical Knowledge for Recognising Textual Entailment

Paper · 2017 · Pascual Martínez-Gómez, Koji Mineshima, Yusuke Miyao et al.

Summary

The paper tackles Recognising Textual Entailment (RTE) with a purely logical pipeline. Sentences are parsed with CCG (C&C and EasyCCG) and composed into Neo-Davidsonian event-semantic formulas. A natural-deduction prover implemented in Coq then checks whether T → H holds. The main obstacle is lexical: the hypothesis often uses different words than the text, so the proof stalls unless someone states the bridging knowledge. Rather than pairing every word in T with every word in H, the authors watch the proof as it runs. When a sub-goal cannot be proved but shares variable instantiations with a premise, they look the two predicates up in WordNet or VerbOcean, add the matching axiom and resume the proof. This method, called SPSA, needs no training or parameter tuning. On SICK it scored the best accuracy among logic-based systems and was competitive with statistical approaches. It also kept precision high, and every axiom it introduces can be inspected.

Key points

  • The semantics is Neo-Davidsonian. Each verb becomes a one-place event predicate, and participants are attached through role functions such as Subj(v) and Obj(v). This lets a single axiom like ∀v(greet(v) → move(v)) connect verbs that take different numbers of arguments.
  • The external knowledge comes only from WordNet and VerbOcean, both chosen for high precision. Abzianidze (2015) added missing facts by hand; this system uses only automatically generated axioms.
  • Each prover call took about 10 ms on average. SPSA made about 10.7 calls per problem versus about 3.7 for the naïve baseline, because the proof restarts after each new axiom. Python overhead outweighed the prover's own time.
  • Typical errors:
    • Out-of-context senses, such as the false axiom wear → break built on the object "gear".
    • Missing relations between sister terms, such as play/strum.
    • Cases that would need string similarity or world knowledge ("on a stage" vs "onstage").

Interesting ideas

  • I1. Knowledge is pulled in only when the proof asks for it: a failed sub-goal triggers the lookup, so lexical relations are checked only where the logic says they matter, which cuts out-of-context matches as sentences grow.
  • I2. A candidate lexical pair counts only if the two predicates share argument instantiations (the same event or entity variable), which gives a cheap structural filter before any resource lookup.
  • I3. Treating verbs as one-place event predicates with separate role functions makes verb-to-verb entailments arity-independent and lets modifiers be dropped soundly.
  • I4. Backtracking over variable and predicate unifications explores several T–H alignments instead of committing to one, which helps when content words repeat.
  • I5. A high-precision symbolic system runs first and hands undecided cases to a statistical judge, which can use the logical forms and axioms as features.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): Readers could attempt the answer derivation first and query WordNet/VerbOcean only for the unresolved goal, logging each injected axiom as part of the explanation.
  • I2 (world): When linking a question predicate to a world fact, require the same entity or event slot before trusting a WordNet hypernym, which would block errors like wear→break.
  • I3 (frames): Frames could keep the event node separate from its VerbNet roles, so verb-class axioms apply regardless of transitivity.
  • I4 (answer): When a question noun matches several entities in the world of a fable, try the alternatives instead of taking the first match.
  • I5 (learn): The strict logical reader could serve as the first witness, and only its "unknown" cases would go to LLM teacher tiers, with the injected axioms recorded as silver data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬National Institute of InformaticsTokyo · Japan21 works · 3,045 citations🔬National Institute of Advanced Industrial Science and TechnologyTsukuba · Japan13 works · 2,747 citations🏛️Japan Science and Technology AgencyTokyo · Japan7 works · 1,755 citations🔬Artificial Intelligence Research CenterTokyo · Japan8 works · 1,556 citations🎓The Graduate University for Advanced Studies, SOKENDAIHayama · Japan3 works · 787 citations

Also on the byline: Ochanomizu University.

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Reasoning · 510 works

Cite

Cite as Pascual Martínez-Gómez, Koji Mineshima, Yusuke Miyao and Daisuke Bekki. 2017. On-demand Injection of Lexical Knowledge for Recognising Textual Entailment. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{martinezgomez2017demand,
  title = {{On-demand Injection of Lexical Knowledge for Recognising Textual Entailment}},
  author = {Pascual Martínez-Gómez, Koji Mineshima, Yusuke Miyao and Daisuke Bekki},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers},
  year = {2017},
  doi = {10.18653/v1/e17-1067},
  url = {https://doi.org/10.18653/v1/e17-1067}
}

References

33 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Lasha Abzianidze. 2015. A tableau prover for natural logic and language. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2492–2502, Lisbon, Portugal, September. Association for Computational Linguistics. find
  2. Islam Beltagy, Cuong Chau, Gemma Boleda, Dan Garrette, Katrin Erk, and Raymond Mooney. 2013. Montague meets Markov: Deep semantics with probabilistic logical form. pages 11–21, June. find
  3. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications, volume 6. Morgan & Claypool Publishers. find
  4. Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, and Raymond J. Mooney. 2016. Representing meaning with a combination of logical and distributional models. Computational Linguistics, 42(4):763–808. ↗
  5. Yubing Dong, Ran Tian, and Yusuke Miyao. 2014. Encoding generalized quantifiers in dependency-based compositional semantics. In Proceedings of the 28th Pacific Asia Conference on Language, Information, and Computation, pages 585–594. find
  6. Johannes Bjerva, Johan Bos, Rob van der Goot, and Malvina Nissim. 2014. The Meaning Factory: Formal semantics for recognizing textual entailment and determining semantic similarity. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 642–646, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. ↗ our card
  7. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 758–764, Atlanta, Georgia, June. Association for Computational Linguistics. our card
  8. Patrick Blackburn and Johan Bos. 2005. Representation and Inference for Natural Language: A First Course in Computational Semantics. CSLI. find
  9. Sergio Jiménez, George Dueñas, Julia Baquero, and Alexander Gelbukh. 2014. UNAL-NLP: Combining soft cardinality features for semantic textual similarity, relatedness and entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 732–742, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. ↗
  10. Johan Bos and Katja Markert. 2005. Recognising textual entailment with logical inference. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 628–635. Association for Computational Linguistics. our card
  11. Johan Bos, Stephen Clark, Mark Steedman, James R Curran, and Julia Hockenmaier. 2004. Wide-coverage semantic representations from a CCG parser. In Proceedings of the 20th international conference on Computational Linguistics, pages 104– 111. Association for Computational Linguistics. find
  12. Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing. Prentice-Hall, Inc. find
  13. Alice Lai and Julia Hockenmaier. 2014. Illinois-LH: A denotational and distributional approach to semantics. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 329–334, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. find
  14. Johan Bos. 2008. Wide-coverage semantic analysis with Boxer. In Proceedings of the 2008 Conference on Semantics in Text Processing, pages 277–286. find
  15. Pierre Castéran and Yves Bertot. 2004. Interactive Theorem Proving and Program Development. Coq’Art: The Calculus of Inductive Constructions. Springer Verlag. ↗
  16. Mike Lewis and Mark Steedman. 2014. A* CCG parsing with a supertag-factored model. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 990– 1000, Doha, Qatar, October. Association for Computational Linguistics. our card
  17. Timothy Chklovski and Patrick Pantel. 2004. Verbocean: Mining the web for fine-grained semantic verb relations. In Dekang Lin and Dekai Wu, editors, Proceedings of EMNLP 2004, pages 33–40, Barcelona, Spain, July. Association for Computational Linguistics. find
  18. Percy Liang, Michael Jordan, and Dan Klein. 2013. Learning dependency-based compositional semantics. Computational Linguistics, 39(2):389–446. ↗
  19. Stephen Clark and James R Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493–552. ↗ our card
  20. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of LREC2014, pages 216–223. our card
  21. Robin Cooper, Richard Crouch, Jan van Eijck, Chris Fox, Josef van Genabith, Jan Jaspers, Hans Kamp, Manfred Pinkal, Massimo Poesio, Stephen Pulman, et al. 1994. FraCaS–a framework for computational semantics. Deliverable, D6. find
  22. Pascual Martı́nez-Gómez, Koji Mineshima, Yusuke Miyao, and Daisuke Bekki. 2016. ccg2lambda: A compositional semantics system. In Proceedings of ACL-2016 System Demonstrations, pages 85–90, Berlin, Germany, August. Association for Computational Linguistics. find
  23. Thierry Coquand and Gerard Huet. 1988. The calculus of constructions. Information and Computation, 76(2-3):95–120. ↗
  24. George A. Miller. 1995. WordNet: A lexical database for English. Commun. ACM, 38(11):39–41, November. our card
  25. Koji Mineshima, Pascual Martı́nez-Gómez, Yusuke Miyao, and Daisuke Bekki. 2015. Higher-order logical inference with compositional semantics. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2055– 2061, Lisbon, Portugal, September. Association for Computational Linguistics. find
  26. Koji Mineshima, Ribeka Tanaka, Pascual Martı́nez-Gómez, Yusuke Miyao, and Daisuke Bekki. 2016. Building compositional semantics and higher-order inference system for a wide-coverage Japanese CCG parser. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2236–2242, Austin, Texas, November. Association for Computational Linguistics. find
  27. Terence Parsons. 1990. Events in the Semantics of English. MIT Press. find
  28. Ellie Pavlick, Johannes Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding semantics to data-driven paraphrasing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL 2015), pages 1512–1522. Association for Computational Linguistics. find
  29. Dag Prawitz. 1965. Natural Deduction – A Proof-Theoretical Study. Almqvist & Wiksell, Stockholm. find
  30. Thomas Proisl, Stefan Evert, Paul Greiner, and Besim Kabashi. 2014. SemantiKLUE: Robust semantic similarity at multiple levels using maximum weight matching. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 532–540, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. find
  31. Mark Steedman. 2000. The Syntactic Process. MIT Press. ↗
  32. Ran Tian, Yusuke Miyao, and Takuya Matsuzaki. 2014. Logical inference on dependency-based compositional semantics. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 79–89, Baltimore, Maryland, June. Association for Computational Linguistics. find
  33. Jiang Zhao, Man Lan, and Tiantian Zhu. 2014. ECNU: Expression- and message-level sentiment orientation classification in twitter using multiple effective features. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 259–264, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gomez-2017-demand-injection-lexical-knowledge-recognising/