Elamur Research⌕
☰
home / papers / 2006 / Learning for Semantic Parsing with Statistical Machine Translation

Learning for Semantic Parsing with Statistical Machine Translation

Summary

The authors address semantic parsing: a natural-language sentence must be transformed into a complete formal meaning representation (MR), which may have a deeply nested structure. They propose WASP (Word Alignment-based Semantic Parsing), a method that treats this task as translation from a natural language into a formal language. Training requires only pairs of the form "sentence — correct MR" and an unambiguous context-free grammar of the MR language. No knowledge of natural-language syntax is needed. First, a statistical word alignment model acquires a bilingual lexicon. To ensure that every generated MR is syntactically well-formed, MRs are represented as linearized sequences of grammar productions. Next, synchronous context-free grammar (SCFG) rules are built from the lexicon, and a probabilistic model selects the most likely derivation. The method was evaluated on the ROBOCUP (CLANG language) and GEOQUERY domains. In terms of precision and recall, WASP generally outperforms approaches that use the same amount of annotation. Moreover, it performs robustly across languages with different word orders. The work showed that machine translation tools transfer well to semantic parsing.

Key points

  • Lexicon and model: GIZA++ (an implementation of IBM Model 5) is used for alignment. Rules are extracted from the K = 10 most probable alignments. The probabilistic derivation model is trained in an unsupervised manner, since the training data contain no gold-standard derivations.
  • Data and evaluation: sentences in the GEOQUERY corpus average 7.48 words, much shorter than in ROBOCUP. Evaluation used 10-fold cross-validation with precision and recall. For ROBOCUP, a translation was counted as correct only if it exactly matched the MR; for GEOQUERY, if the query returned the same answer.
  • Comparison: WASP outperforms SILT and COCKTAIL, which learn from the same data. COCKTAIL could not handle the long ROBOCUP sentences and crashed on large training sets due to running out of memory. WASP's advantage over SILT lies primarily in recall. Methods that use additional annotation (SCISSOR, the Zettlemoyer and Collins parser) perform better.
  • Multilinguality: 250 GEOQUERY queries were translated into Spanish, Japanese, and Turkish, thus covering both SVO and SOV word orders. Results are similar across languages. Recall for Turkish is slightly lower, most likely due to its considerably larger vocabulary.
  • Speed: training with IIS took 47 minutes for ROBOCUP and 83 minutes for GEOQUERY. Preliminary experiments indicate that L-BFGS cuts training time by more than half without loss of accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics Machine translation Semantic parsing & meaning representation

Institutions

🎓The University of Texas at AustinAustin · United States51 works · 52,106 citations

Cite

Cite as Yuk Wah Wong and Raymond Mooney. 2006. Learning for Semantic Parsing with Statistical Machine Translation. Proceedings of the Human Language Technology Conference of the NAACL, Main Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{wong2006learning,
  title = {{Learning for Semantic Parsing with Statistical Machine Translation}},
  author = {Yuk Wah Wong and Raymond Mooney},
  booktitle = {Proceedings of the Human Language Technology Conference of the NAACL, Main Conference},
  year = {2006},
  doi = {10.3115/1220835.1220891},
  url = {https://doi.org/10.3115/1220835.1220891}
}

References

24 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. V. Aho and J. D. Ullman. 1972. The Theory of Parsing, Translation, and Compiling. Prentice Hall, Englewood Cliffs, NJ. find
  2. J. F. Allen. 1995. Natural Language Understanding (2nd Ed.). Benjamin/Cummings, Menlo Park, CA. find
  3. I. Androutsopoulos, G. D. Ritchie, and P. Thanisch. 1995. Natural language interfaces to databases: An introduction. Journal of Natural Language Engineering, 1(1):29–81. find
  4. P. F. Brown, V. J. Della Pietra, S. A. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–312, June. find
  5. S. Chen and R. Rosenfeld. 1999. A Gaussian prior for smoothing maximum entropy models. Technical report, Carnegie Mellon University, Pittsburgh, PA. find
  6. M. Chen et al. 2003. Users manual: RoboCup soccer server manual for soccer server version 7.07 and later. Available at http://sourceforge.net/projects/sserver/. ↗
  7. D. Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. of ACL-05, pages 263–270, Ann Arbor, MI, June. our card
  8. R. Ge and R. J. Mooney. 2005. A statistical semantic parser that integrates syntax and semantics. In Proc. of CoNLL-05, pages 9–16, Ann Arbor, MI, July. find
  9. R. J. Kate, Y. W. Wong, and R. J. Mooney. 2005. Learning to transform natural to formal languages. In Proc. of AAAI-05, pages 1062–1068, Pittsburgh, PA, July. find
  10. G. Kuhlmann, P. Stone, R. J. Mooney, and J. W. Shavlik. 2004. Guiding a reinforcement learner with natural language advice: Initial results in RoboCup soccer. In Proc. of the AAAI-04 Workshop on Supervisory Control of Learning and Adaptive Systems, San Jose, CA, July. find
  11. K. Macherey, F. J. Och, and H. Ney. 2001. Natural language understanding using statistical machine translation. In Proc. of EuroSpeech-01, pages 2205–2208, Aalborg, Denmark. find
  12. I. D. Melamed. 2004. Statistical machine translation by parsing. In Proc. of ACL-04, pages 653–660, Barcelona, Spain. find
  13. S. Miller, D. Stallard, R. Bobrow, and R. Schwartz. 1996. A fully statistical approach to natural language interfaces. In Proc. of ACL-96, pages 55–61, Santa Cruz, CA. find
  14. Y. Miyao and J. Tsujii. 2002. Maximum entropy estimation for feature forests. In Proc. of HLT-02, San Diego, CA, March. find
  15. J. Nocedal. 1980. Updating quasi-Newton matrices with limited storage. Mathematics of Computation, 35(151):773–782, July. find
  16. F. J. Och and H. Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  17. K. A. Papineni, S. Roukos, and R. T. Ward. 1997. Feature-based language understanding. In Proc. of EuroSpeech-97, pages 1435–1438, Rhodes, Greece. find
  18. S. Riezler, D. Prescher, J. Kuhn, and M. Johnson. 2000. Lexicalized stochastic modeling of constraint-based grammars using log-linear measures and EM training. In Proc. of ACL-00, pages 480–487, Hong Kong. find
  19. A. Stolcke. 1995. An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2):165–201. our card
  20. L. R. Tang and R. J. Mooney. 2001. Using multiple clause constructors in inductive logic programming for semantic parsing. In Proc. of ECML-01, pages 466– 477, Freiburg, Germany. find
  21. Y. W. Wong. 2005. Learning for semantic parsing using statistical machine translation techniques. Technical Report UT-AI-05-323, Artificial Intelligence Lab, University of Texas at Austin, Austin, TX, October. find
  22. K. Yamada and K. Knight. 2001. A syntax-based statistical translation model. In Proc. of ACL-01, pages 523–530, Toulouse, France. our card
  23. J. M. Zelle and R. J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proc. of AAAI-96, pages 1050–1055, Portland, OR, August. our card
  24. L. S. Zettlemoyer and M. Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proc. of UAI-05, Edinburgh, Scotland, July. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wong-2006-learning-semantic-parsing-statistical-machine/