Elamur Research⌕
☰
home / papers / 2012 / Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars

Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars

Summary

The authors address the task of semantic parsing: a natural-language sentence must be mapped to a lambda-calculus formula that expresses its meaning. The main difficulty is that the training data contain only "sentence — logical form" pairs. They include no intermediate syntactic derivations, so the model must work out on its own which word corresponds to which predicate. The proposed algorithm simultaneously induces a CCG (combinatory categorial grammar) and trains a log-linear model similar to a CRF. This model defines probabilities over the possible parses of a sentence. The GENLEX procedure generates a large number of candidate lexical items, and the superfluous ones are pruned during training. The method was evaluated on queries to two databases: one on US geography (Geo880) and one on job listings (Jobs640). On both datasets, precision exceeded 95% and recall was 79%. In precision, the method outperformed the COCKTAIL system, even though that system was given a hand-built lexicon. The work showed that a natural-language interface to a database can be learned with minimal annotation. The approach later became one of the baselines in semantic parsing.

Key points

  • Data: Geo880 was split into 600 training and 280 test examples, and Jobs640 into 500 and 140. The authors manually rewrote the original Prolog annotation in lambda calculus.
  • Results: precision above 95% and recall of 79% on both domains. Only a fully correct logical form was counted as correct. Compared with COCKTAIL, precision is higher and recall only slightly lower.
  • Compact lexicon: on Geo880, GENLEX generated on average 393.8 candidates per example, and 5.1 remained after pruning. On Jobs640, this number dropped from 697.1 to 6.6.
  • Reason for the lower recall: the system failed to parse about 9.3% of the training sentences in Geo880 and 8.7% in Jobs640. These are mostly complex constructions, for example with a preposition moved to the beginning of the sentence, for which GENLEX lacks the necessary rules.
  • Formalism: CCG with application, composition and type-raising rules. Lexical items can consist of several words; for example, "the Mississippi" denotes a river.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Syntax & grammar formalisms Semantic parsing & meaning representation

Cite

Cite as Luke S. Zettlemoyer and Michael Collins. 2012. Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars. arXiv:1207.1420.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1207.1420 · arXiv 1207.1420

@misc{zettlemoyer2012learning,
  title = {{Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars}},
  author = {Luke S. Zettlemoyer and Michael Collins},
  year = {2012},
  doi = {10.48550/arxiv.1207.1420},
  eprint = {1207.1420},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1207.1420}
}

References

23 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Androutsopoulos, I., Ritchie, G., & Thanisch, P. (1995). Natural language interfaces to databases–an introduction. Journal of Language Engineering, 1(1), 29– 81. find
  2. Baker, J. (1979). Trainable grammars for speech recognition. In Speech Communication Papers for the 97th Meeting of the Acoustical Society of America. find
  3. Bos, J., Clark, S., Steedman, M., Curran, J. R., & Hockenmaier, J. (2004). Wide-coverage semantic representations from a CCG parser. In Proceedings of the 20th International Conference on Computational Linguistics, pp. 1240–1246. find
  4. Carpenter, B. (1997). Type-Logical Semantics. The MIT Press. find
  5. Clark, S., & Curran, J. R. (2003). Log-linear models for wide-coverage CCG parsing. In Proceedings of the SIGDAT Conference on Empirical Methods in Natural Language Processing. find
  6. He, Y., & Young, S. (2004). Semantic processing using the hidden vector state model. Computer Speech and Language. find
  7. Johnson, M., Geman, S., Canon, S., Chi, Z., & Riezler, S. (1999). Estimators for stochastic “unification-based” grammars. In Proceedings of the Association for Computational Linguistics, pp. 535–541. find
  8. Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. find
  9. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. find
  10. Manning, C. D., & Schutze, H. (1999). Foundations of statistical natural language processing. The MIT Press. find
  11. Miller, S., Stallard, D., Bobrow, R. J., & Schwartz, R. L. (1996). A fully statistical approach to natural language interfaces. In Proceedings of the Association for Computational Linguistics, pp. 55–61. find
  12. Papineni, K. A., Roukos, S., & Ward, T. R. (1997). Feature-based language understanding. In Proceedings of European Conference on Speech Communication and Technology. find
  13. Popescu, A.-M., Armanasu, A., Etzioni, O., Ko, D., & Yates, A. (2004). Modern natural language interfaces to databases: Composing statistical parsing with semantic tractability. In Proceedings of the 20th International Conference on Computational Linguistics. find
  14. Ramaswamy, G., & Kleindienst, J. (2000). Hierarchical feature-based translation for scalable natural language understanding. In Proceedings of 6th International Conference on Spoken Language Processing. find
  15. Ratnaparkhi, A., Roukos, S., & Ward, R. T. (1994). A maximum entropy model for parsing. In Proceedings of the International Conference on Spoken Language Processing. find
  16. Steedman, M. (1996). Surface Structure and Interpretation. The MIT Press. find
  17. Steedman, M. (2000). The Syntactic Process. The MIT Press. find
  18. Tang, L. R., & Mooney, R. J. (2001). Using multiple clause constructors in inductive logic programming for semantic parsing. In Proceedings of the 12th European Conference on Machine Learning, pp. 466–477. find
  19. Taskar, B., Guestrin, C., & Koller, D. (2003). Max-margin markov networks. In Neural Information Processing Systems. find
  20. Taskar, B., Klein, D., Collins, M., Koller, D., & Manning, C. (2004). Max-margin parsing. In Proceedings of the SIGDAT Conference on Empirical Methods in Natural Language Processing. find
  21. Thompson, C. A., & Mooney, R. J. (2002). Acquiring word-meaning mappings for natural language interfaces. Journal of Artificial Intelligence Research, 18. find
  22. Watkinson, S., & Manandhar, S. (1999). Unsupervised lexical learning with categorial grammars using the LLL corpus. In Proceedings of the 1st Workshop on Learning Language in Logic, pp. 16–27. find
  23. Zelle, J. M., & Mooney, R. J. (1996). Learning to parse database queries using inductive logic programming. In Proceedings of the 14th National Conference on Artificial Intelligence. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zettlemoyer-2012-learning-map-sentences-logical-form/