Elamur Research⌕
☰
home / papers / 2015 / Compositional Semantic Parsing on Semi-Structured Tables

Compositional Semantic Parsing on Semi-Structured Tables

Summary

The authors combine two goals of semantic parsing that previously had to be traded off against each other: a broad knowledge source and complex multi-step questions. Early systems understood complex questions, but only over small single-domain databases. Later systems worked with large knowledge bases such as Freebase, yet answered only simple questions. The authors propose a new task: answering complex questions over HTML tables from Wikipedia, while training the system only on question–answer pairs. The tables in the test set do not appear during training, so a relation lexicon cannot be compiled in advance. Each table is deterministically converted into a knowledge graph with nodes for normalized numbers and dates. A broad-coverage grammar builds candidate logical forms in the lambda DCS language. A log-linear model ranks the candidates, and beam search with type- and denotation-based pruning keeps the combinatorial explosion in check. The open WikiTableQuestions dataset was created for evaluation. The system clearly outperforms the baseline approaches and has become a starting point for research on question answering over tables.

Key points

  • WikiTableQuestions: 22,033 questions over 2,108 Wikipedia tables, each table having at least 8 rows and 5 columns. The questions were written and verified by workers on Amazon Mechanical Turk. An answer was kept only if two workers agreed on it, and about 69% of the questions remained after this filter. 20% of the tables were held out for testing.
  • Accuracy: 37.1% on tables the system did not see during training. An information-retrieval baseline achieves 12.7%, and a simple semantic parser achieves 24.3%.
  • Method: a knowledge graph (rows, cells, and auxiliary Next and Index edges), lambda DCS logical forms, log-linear ranking of candidates, and training with AdaGrad and L1 regularization.
  • Coverage: 79% of the questions can be manually translated into a logical form, and the system generates the correct form among its candidates in 53.5% of cases.
  • Error analysis: 29% are caused by normalization and types, 25% by entity linking to the text, another 25% by candidate ranking, and 21% by question types the system does not handle.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Semantic parsing & meaning representation

Cite

Cite as Panupong Pasupat and Percy Liang. 2015. Compositional Semantic Parsing on Semi-Structured Tables. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1142 · arXiv 1508.00305

@inproceedings{pasupat2015compositional,
  title = {{Compositional Semantic Parsing on Semi-Structured Tables}},
  author = {Panupong Pasupat and Percy Liang},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1142},
  eprint = {1508.00305},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/v1/p15-1142}
}

References

33 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. I. Androutsopoulos, G. D. Ritchie, and P. Thanisch. 1995. Natural language interfaces to databases – an introduction. Journal of Natural Language Engineering, 1:29–81. find
  2. J. Berant and P. Liang. 2014. Semantic parsing via paraphrasing. In Association for Computational Linguistics (ACL). our card
  3. J. Berant, A. Chou, R. Frostig, and P. Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Empirical Methods in Natural Language Processing (EMNLP). our card
  4. M. J. Cafarella, A. Halevy, D. Z. Wang, E. Wu, and Y. Zhang. 2008. WebTables: exploring the power of tables on the web. In Very Large Data Bases (VLDB), pages 538–549. find
  5. Q. Cai and A. Yates. 2013. Large-scale semantic parsing via schema matching and lexicon extension. In Association for Computational Linguistics (ACL). find
  6. J. Duchi, E. Hazan, and Y. Singer. 2010. Adaptive subgradient methods for online learning and stochastic optimization. In Conference on Learning Theory (COLT). find
  7. A. Fader, L. Zettlemoyer, and O. Etzioni. 2013. Paraphrase-driven learning for open question answering. In Association for Computational Linguistics (ACL). our card
  8. A. Fader, L. Zettlemoyer, and O. Etzioni. 2014. Open question answering over curated and extracted knowledge bases. In International Conference on Knowledge Discovery and Data Mining (KDD), pages 1156–1165. find
  9. H. Gonzalez, A. Y. Halevy, C. S. Jensen, A. Langen, J. Madhavan, R. Shapley, W. Shen, and J. Goldberg-Kidon. 2010. Google fusion tables: web-centered data management and collaboration. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pages 1061–1066. find
  10. R. Gupta and S. Sarawagi. 2009. Answering table augmentation queries from unstructured lists on the web. In Very Large Data Bases (VLDB), number 1, pages 289–300. find
  11. H. Ji and R. Grishman. 2011. Knowledge base population: Successful approaches and challenges. In Association for Computational Linguistics (ACL), pages 1148–1158. find
  12. J. Krishnamurthy and T. Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Association for Computational Linguistics (TACL), 1:193– 206. find
  13. T. Kwiatkowski, L. Zettlemoyer, S. Goldwater, and M. Steedman. 2011. Lexical generalization in CCG grammar induction for semantic parsing. In Empirical Methods in Natural Language Processing (EMNLP), pages 1512–1523. our card
  14. T. Kwiatkowski, E. Choi, Y. Artzi, and L. Zettlemoyer. 2013. Scaling semantic parsers with on-the-fly ontology matching. In Empirical Methods in Natural Language Processing (EMNLP). our card
  15. P. Liang. 2013. Lambda dependency-based compositional semantics. arXiv. find
  16. G. Limaye, S. Sarawagi, and S. Chakrabarti. 2010. Annotating and searching web tables using entities, types and relationships. In Very Large Data Bases (VLDB), volume 3, pages 1338–1347. find
  17. Masaum, M. Schmitz, R. Bart, S. Soderland, and O. Etzioni. 2012. Open language learning for information extraction. In Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP/CoNLL), pages 523– 534. find
  18. P. Pasupat and P. Liang. 2014. Zero-shot entity extraction from web pages. In Association for Computational Linguistics (ACL). find
  19. R. Pimplikar and S. Sarawagi. 2012. Answering table queries on the web using column keywords. In Very Large Data Bases (VLDB), volume 5, pages 908– 919. find
  20. A. Popescu, O. Etzioni, and H. Kautz. 2003. Towards a theory of natural language interfaces to databases. In International Conference on Intelligent User Interfaces (IUI), pages 149–157. find
  21. P. Price. 1990. Evaluation of spoken language systems: The ATIS domain. In Proceedings of the Third DARPA Speech and Natural Language Workshop, pages 91–95. find
  22. S. Reddy, M. Lapata, and M. Steedman. 2014. Large-scale semantic parsing without question-answer pairs. Transactions of the Association for Computational Linguistics (TACL), 2(10):377–392. find
  23. Z. Syed, T. Finin, V. Mulwad, and A. Joshi. 2010. Exploiting a web of semantic data for interpreting tables. In Proceedings of the Second Web Science Conference. find
  24. C. Unger and P. Cimiano. 2011. Pythia: compositional meaning construction for ontology-based question answering on the semantic web. In Proceedings of the 16th international conference on Natural language processing and information systems, pages 153–160. find
  25. C. Unger, L. Bühmann, J. Lehmann, A. Ngonga, D. Gerber, and P. Cimiano. 2012. Template-based question answering over RDF data. In World Wide Web (WWW), pages 639–648. find
  26. P. Venetis, A. Halevy, J. Madhavan, M. Paşca, W. Shen, F. Wu, G. Miao, and C. Wu. 2011. Recovering semantics of tables on the web. In Very Large Data Bases (VLDB), volume 4, pages 528–538. find
  27. Y. Wang, L. Deng, and A. Acero. 2011. Semantic frame-based spoken language understanding. Spoken Language Understanding: Systems for Extracting Semantic Information from Speech, pages 41–91. find
  28. Y. W. Wong and R. J. Mooney. 2007. Learning synchronous grammars for semantic parsing with lambda calculus. In Association for Computational Linguistics (ACL), pages 960–967. our card
  29. Y. W. Wong, D. Widdows, T. Lokovic, and K. Nigam. 2009. Scalable attribute-value extraction from semi-structured text. In IEEE International Conference on Data Mining Workshops, pages 302–307. find
  30. F. Wu and D. S. Weld. 2010. Open information extraction using Wikipedia. In Association for Computational Linguistics (ACL), pages 118–127. our card
  31. M. Zelle and R. J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Association for the Advancement of Artificial Intelligence (AAAI), pages 1050–1055. our card
  32. L. S. Zettlemoyer and M. Collins. 2007. Online learning of relaxed CCG grammars for parsing to logical form. In Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP/CoNLL), pages 678–687. find
  33. Z. Zhang, K. Q. Zhu, H. Wang, and H. Li. 2013. Automatic extraction of top-k lists from the web. In International Conference on Data Engineering. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pasupat-2015-compositional-semantic-parsing-semi-structured/