Elamur Research⌕
☰
home / papers / 2013 / Scaling Semantic Parsers with On-the-Fly Ontology Matching

Scaling Semantic Parsers with On-the-Fly Ontology Matching

Summary

The authors address the problem of scaling semantic parsers to large open-domain knowledge bases such as Freebase. The main obstacle here is ontological mismatch: words in a question often have no direct counterpart among the constants of the knowledge base. For example, Freebase has no separate concept of "daughter"; it only has gender and parenthood. The parser works in two stages. First, a probabilistic CCG, together with word-class information from Wiktionary, builds an underspecified logical form that is domain-independent and mirrors the linguistic structure of the sentence. Then a learned ontology matching model transforms this form on the fly into a query over the specific schema of the knowledge base. Training requires only question–answer pairs, with no annotated logical forms and no induced domain lexicon. On the Freebase QA corpus the method beats the best results reported before it, and on GeoQuery it performs on par with the strongest systems. This matters because a single grammar can be reused across domains, and the system infers the meanings of unseen words on its own.

Key points

  • Data: GeoQuery (a small geographic ontology with complex compositional questions) and Freebase QA (FQ) from Cai and Yates (2013a). There is also a cross-domain evaluation with FQ split by topic (sports, film, business).
  • On the FQ test set: recall 68.0, precision 76.7, F1 72.1 versus 59 / 67 / 63 for CY13, i.e., a recall gain of about 9 points. In the cross-domain setting, F1 is 71.5 versus 65.
  • On GeoQuery, recall is 89.0. This is better than FUBL (88.6) and DCS (87.9), but slightly below DCS with a hand-crafted seed lexicon (91.1).
  • Ablation with 5-fold cross-validation: removing the knowledge-base features drops F1 from 70.3 to 62.1, because the model starts generating queries with empty answers. Without the Wiktionary features, F1 decreases only to 68.9.
  • The domain-independent grammar relies on 56 underspecified lexical categories. Error analysis on FQ: lexical ambiguity ≈35%, structural matching errors ≈30%, parsing failures ≈20%, knowledge base incompleteness ≈10%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Knowledge graphs & ontologies Semantic parsing & meaning representation

Cite

Cite as Tom Kwiatkowski, Eunsol Choi, Yoav Artzi and Luke Zettlemoyer. 2013. Scaling Semantic Parsers with On-the-Fly Ontology Matching. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kwiatkowski2013scaling,
  title = {{Scaling Semantic Parsers with On-the-Fly Ontology Matching}},
  author = {Tom Kwiatkowski, Eunsol Choi, Yoav Artzi and Luke Zettlemoyer},
  booktitle = {Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  url = {https://aclanthology.org/D13-1161/}
}

References

38 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alshawi, H. (1992). The core language engine. The MIT Press. find
  2. Artzi, Y. and Zettlemoyer, L. (2011). Bootstrapping semantic parsers from conversations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  3. Artzi, Y. and Zettlemoyer, L. (2013). Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association for Computational Linguistics, 1(1):49–62. our card
  4. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data. find
  5. Bos, J. (2008). Wide-coverage semantic analysis with boxer. In Proceedings of the Conference on Semantics in Text Processing. find
  6. Cai, Q. and Yates, A. (2013a). Large-scale semantic parsing via schema matching and lexicon extension. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. find
  7. Cai, Q. and Yates, A. (2013b). Semantic parsing freebase: Towards open-domain semantic parsing. In Proceedings of the Joint Conference on Lexical and Computational Semantics. find
  8. Chen, D. and Mooney, R. (2011). Learning to interpret natural language navigation instructions from observations. In Proceedings of the National Conference on Artificial Intelligence. find
  9. Clark, S. and Curran, J. (2007). Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493– 552. our card
  10. Clarke, J., Goldwasser, D., Chang, M., and Roth, D. (2010). Driving semantic parsing from the world’s response. In Proceedings of the Conference on Computational Natural Language Learning. find
  11. Davidson, D. (1967). The logical form of action sentences. Essays on actions and events, pages 105– 148. find
  12. Doan, A., Madhavan, J., Domingos, P., and Halevy, A. (2004). Ontology matching: A machine learning approach. In Handbook on ontologies. Springer. find
  13. Euzenat, J., Euzenat, J., Shvaiko, P., et al. (2007). Ontology matching. Springer. find
  14. Fader, A., Zettlemoyer, L., and Etzioni, O. (2013). Paraphrase-driven learning for open question answering. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. our card
  15. Goldwasser, D. and Roth, D. (2011). Learning from natural instructions. In Proceedings of the International Joint Conference on Artificial Intelligence. find
  16. Grosz, B. J., Appelt, D. E., Martin, P. A., and Pereira, F. (1987). TEAM: An experiment in the design of transportable natural language interfaces. Artificial Intelligence, 32(2):173–243. find
  17. Hobbs, J. R. (1985). Ontological promiscuity. In Proceedings of the Annual Meeting on Association for Computational Linguistics. find
  18. Jones, B. K., Johnson, M., and Goldwater, S. (2012). Semantic parsing with bayesian tree transducers. In Proceedings of the 50th Annual Meeting of the Association of Computational Linguistics. find
  19. Kate, R. and Mooney, R. (2006). Using string-kernels for learning semantic parsers. In Proceedings of the Conference of the Association for Computational Linguistics. find
  20. Krishnamurthy, J. and Kollar, T. (2013). Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Association for Computational Linguistics, 1(2). find
  21. Krishnamurthy, J. and Mitchell, T. (2012). Weakly supervised training of semantic parsers. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. find
  22. Kushman, N. and Barzilay, R. (2013). Using semantic unification to generate regular expressions from natural language. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics. find
  23. Kwiatkowski, T., Goldwater, S., Zettlemoyer, L., and Steedman, M. (2012). A probabilistic model of syntactic and semantic acquisition from child-directed utterances and their meanings. Proceedings of the Conference of the European Chapter of the Association of Computational Linguistics. our card
  24. Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., and Steedman, M. (2010). Inducing probabilistic CCG grammars from logical form with higher-order unification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  25. Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., and Steedman, M. (2011). Lexical generalization in CCG grammar induction for semantic parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. our card
  26. Liang, P., Jordan, M., and Klein, D. (2011). Learning dependency-based compositional semantics. In Proceedings of the Conference of the Association for Computational Linguistics. find
  27. Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., and Fox, D. (2012). A joint model of language and perception for grounded attribute learning. In Proceedings of the International Conference on Machine Learning. find
  28. Muresan, S. (2011). Learning for deep language understanding. In Proceedings of the International Joint Conference on Artificial Intelligence. find
  29. Steedman, M. (1996). Surface Structure and Interpretation. The MIT Press. find
  30. Steedman, M. (2000). The Syntactic Process. The MIT Press. find
  31. Unger, C., Bühmann, L., Lehmann, J., Ngonga Ngomo, A., Gerber, D., and Cimiano, P. (2012). Template-based question answering over RDF data. In Proceedings of the International Conference on World Wide Web. find
  32. Wong, Y. and Mooney, R. (2007). Learning synchronous grammars for semantic parsing with lambda calculus. In Proceedings of the Conference of the Association for Computational Linguistics. our card
  33. Yahya, M., Berberich, K., Elbassuoni, S., Ramanath, M., Tresp, V., and Weikum, G. (2012). Natural language questions for the web of data. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  34. Zelle, J. and Mooney, R. (1996). Learning to parse database queries using inductive logic programming. In Proceedings of the National Conference on Artificial Intelligence. our card
  35. Zettlemoyer, L. and Collins, M. (2005). Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Conference on Uncertainty in Artificial Intelligence. our card
  36. Zettlemoyer, L. and Collins, M. (2007). Online learning of relaxed CCG grammars for parsing to logical form. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. find
  37. Zettlemoyer, L. and Collins, M. (2009). Learning context-dependent mappings from sentences to logical form. In Proceedings of the Joint Conference of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing. our card
  38. Zhang, C., Hoffmann, R., and Weld, D. S. (2012). Ontological smoothing for relation extraction with minimal supervision. In Proceeds of the Conference on Artificial Intelligence. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kwiatkowski-2013-scaling-semantic-parsers-fly-ontology/