Elamur Research⌕
☰
home / papers / 2021 / Knowledge-driven Natural Language Understanding of English Text and its Applications

Knowledge-driven Natural Language Understanding of English Text and its Applications

Paper · 2021 · Kinjal Basu, Sarat Varanasi, Farhad Shakerin et al.

Summary

The paper argues that systems which answer questions or hold conversations should first turn text into an explicit meaning representation and then reason over it, rather than rely on word correlations. The authors treat English like a programming language with a denotational semantics. A Stanford CoreNLP parse tree is the syntax. VerbNet's primitive predicates act as the semantic algebra. VerbNet's frames work as the valuation function. Their algorithm partially matches a sentence's parse tree against the frame syntax of the verb's VerbNet classes. It then binds the thematic roles (AGENT, THEME, …) and emits the frame semantics as answer set programming (ASP) facts. Reusable, hand-written commonsense rules are added, and the s(CASP) engine runs the reasoning and produces justifications. Two applications are built on this: SQuARE, a reading-comprehension QA system, and StaCACK, a goal-oriented chatbot. Both reach essentially perfect accuracy on the bAbI benchmarks, beating neural baselines, and both explain their answers in natural language.

Key points

  • Pipeline: CoreNLP parse tree → partial match to VerbNet frame syntax → grounded thematic roles → ASP facts → s(CASP) query with commonsense rules → answer plus proof-tree justification.
  • s(CASP) is goal-directed and avoids grounding the program. It visits only the parts of the knowledge base a query needs and returns a natural-language proof. The authors present these properties as the key to scalability.
  • On the 20 bAbI QA tasks, SQuARE scores 100% on almost every task, with 99.8 on Three Arg. Relation and 98.2 on Indefinite Knowledge. MemNN (AM+NG+NL) averages 94, falling to 60 on Positional Reasoning and 35 on Path Finding.
  • On bAbI dialog Tasks 1–5 and their OOV variants, StaCACK gets 100% both per response and per dialog. On Task 5 (OOV), Mem2Seq scores 84.5 (2.3) and BoSsNET 91.7 (18.5).
  • Nonmonotonic reasoning, implemented as negation-as-failure, handles defaults, exceptions and preferences. Classical negation separately marks facts that are known to be false.

Interesting ideas

  • I1. VerbNet works as a ready-made table from syntax patterns to meanings: match the dependency structure against a class's frame syntax, fill the role slots, and copy out its semantic predicates
  • I2. Framing NLU as denotational semantics (syntax, semantic algebra, valuation function) gives a clean, compositional contract for each layer
  • I3. Goal-directed reasoning explores only the facts relevant to a question and hands back the proof tree as a human-readable justification
  • I4. Defaults with exceptions through negation-as-failure, kept separate from explicit "definitely false" facts, separates "not stated" from "denied"
  • I5. Reporting per-dialog accuracy next to per-response accuracy, plus OOV splits, exposes failures that average scores hide, as in BoSsNET's 18.5% per-dialog result

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could adopt this "partial match + role binding" as its main route from UD trees to VerbNet-based who-did-what-to-whom frames, with the matched frame recorded as the explanation.
  • I2 (world): the world builder could use VerbNet predicates as its fixed primitive vocabulary, so frames compile into entity, state and holding facts in one well-defined algebra.
  • I3 (answer): the readers answering fable QA or school tests could search backward from the question over the text's world and print the derivation as the answer's explanation.
  • I4 (world): the compiled knowledge layer could hold commonsense defaults that a text can override, while preserving the unknown/false distinction for honest answers.
  • I5 (learn): evaluation could add whole-document or whole-problem correctness and out-of-vocabulary splits as stricter controls beside per-item scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Texas at DallasRichardson · United States47 works · 5,363 citations

Also on the byline: Universidad Rey Juan Carlos · LinkedIn.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Kinjal Basu, Sarat Varanasi, Farhad Shakerin, Joaquin Arias and Gopal Gupta. 2021. Knowledge-driven Natural Language Understanding of English Text and its Applications. arXiv:2101.11707.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2101.11707 · arXiv 2101.11707

@misc{basu2021knowledge,
  title = {{Knowledge-driven Natural Language Understanding of English Text and its Applications}},
  author = {Kinjal Basu, Sarat Varanasi, Farhad Shakerin, Joaquin Arias and Gopal Gupta},
  year = {2021},
  doi = {10.48550/arxiv.2101.11707},
  eprint = {2101.11707},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.11707}
}

References

27 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arias, J.; Carro, M.; Salazar, E.; Marple, K.; and Gupta, G. 2018. Constraint answer set programming without grounding. TPLP 18(3-4):337– 354. ↗
  2. Arias, J.; Carro, M.; Chen, Z.; and Gupta, G. 2020. Justifications for goal-directed constraint answer set programming. arXiv preprint arXiv:2009.10238. ↗
  3. Baral, C. 2003. Knowledge representation, reasoning and declarative problem solving. Cambridge University Press. ↗
  4. Basu, K.; Varanasi, S. C.; Shakerin, F.; and Gupta, G. 2020. SQuARE: Semantics-based Question Answering and Reasoning Engine. In Proc. 36th ICLP (Technical Communications), volume 325 of EPTCS, 73–86. ↗
  5. Basu, K.; Shakerin, F.; and Gupta, G. 2020. AQuA: ASP-Based Visual Question Answering. In Practical Aspects of Declarative Languages, 57–72. Cham: Springer International Publishing. find
  6. Bordes, A.; Boureau, Y.-L.; and Weston, J. 2016. Learning end-to-end goal-oriented dialog. arXiv preprint arXiv:1605.07683. ↗
  7. Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165. ↗
  8. Chan, A.; Ma, L.; Juefei-Xu, F.; Xie, X.; Liu, Y.; and Ong, Y. S. 2018. Metamorphic relation based adversarial attacks on differentiable neural computer. arXiv preprint arXiv:1809.02444. ↗
  9. Colby, K. M.; Weber, S.; and Hilf, F. D. 1971. Artificial paranoia. Artificial Intelligence 2(1):1–25. find
  10. Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  11. Gelfond, M., and Kahl, Y. 2014. Knowledge representation, reasoning, and the design of intelligent agents: The answer-set programming approach. Cambridge University Press. find
  12. Gelfond, M., and Lifschitz, V. 1988. The stable model semantics for logic programming. In ICLP/SLP, volume 88, 1070–1080. find
  13. Honnibal, M., and Montani, I. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
  14. Kipper, K.; Korhonen, A.; Ryant, N.; and Palmer, M. 2008. A large-scale classification of english verbs. Language Resources and Evaluation 42(1):21–40. ↗
  15. Kumar, A., et al. 2016. Ask me anything: Dynamic memory networks for natural language processing. In ICML, 1378–1387. ↗
  16. Levin, B. 1993. English verb classes and alternations: A preliminary investigation. U. Chicago Press. find
  17. Lierler, Y.; Inclezan, D.; and Gelfond, M. 2017. Action languages and question answering. In IWCS 2017—12th International Conference on Computational Semantics—Short papers. find
  18. Ling, G. 2018. From Narrative Text to VerbNet-based DRSes: System Text2DRS. find
  19. Madotto, A.; Wu, C.-S.; and Fung, P. 2018. Mem2Seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. arXiv preprint arXiv:1804.08217. ↗
  20. Manning, C. D.; Surdeanu, M.; Bauer, J.; Finkel, J.; Bethard, S. J.; and McClosky, D. 2014. The Stanford CoreNLP NLP toolkit. In ACL System Demonstrations, 55–60. find
  21. McFate, C. 2010. Expanding verb coverage in Cyc with VerbNet. In Proceedings of the ACL 2010 Student Research Workshop, 61–66. find
  22. Pendharkar, D., and Gupta, G. 2019. An ASP Based Approach to Answering Questions for Natural Language Text. In Proc. Practical Aspects of Declarative Languages, Springer LNCS 11372, pp. 46-63, volume 11372 of Lecture Notes in Computer Science, 46–63. Springer. find
  23. Raghu, D.; Gupta, N.; et al. 2018. Disentangling language and knowledge in task-oriented dialogs. arXiv preprint arXiv:1805.01216. ↗
  24. Schmidt, D. A. 1986. Denotational semantics: a methodology for language development, William C. Brown Publishers, Dubuque, IA, USA. find
  25. Schmitz, M.; Soderland, S.; Bart, R.; Etzioni, O.; et al. 2012. Open language learning for information extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 523–534. find
  26. Weizenbaum, J. 1966. ELIZA—a computer program for the study of natural language communication between man and machine. CACM 9(1):36–45. ↗
  27. Weston, J., et al. 2015. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks. arXiv preprint arXiv:1502.05698. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/basu-2021-knowledge-driven-natural-language-understanding/