Elamur Research⌕
☰
home / papers / 2012 / A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings

A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings

Summary

The authors model how a child might acquire grammar and word meanings simultaneously when hearing utterances paired with several possible interpretations of the situation, without knowing which one is correct. As data they use adult child-directed speech from the CHILDES corpus. Each utterance is assigned a set of candidate meanings in the form of logical formulas (lambda calculus). Neither the meanings of individual words nor syntactic parses are given in these data. The model uses the CCG formalism and a generative Bayesian model of parses with Dirichlet process priors. Learning proceeds incrementally via online variational Bayesian EM: the model sees each example only once and is given no language-specific knowledge. On new sentences it recovers their meaning more accurately than the strong semantic parser of Kwiatkowski et al. (2010) when both systems are trained under comparable conditions. Moreover, the model reproduces two characteristics of child language acquisition: "fast mapping" and an abrupt, rather than gradual, transition to correct word order. The authors thereby show that statistical learning models can account for sudden changes in children's linguistic competence.

Key points

  • Data: pairs of "utterance — several candidate meanings" from the CHILDES corpus (MacWhinney, 2000). The authors call the contextual ambiguity "propositional uncertainty".
  • Method: probabilistic CCG. The probability of a parse equals the product of the probabilities of its syntactic and lexical productions. A DP-prior is placed on each production. The lexicon is read off from all admissible parses, and the parameters are estimated via online VBEM in a single pass over the data.
  • Comparison: the model outperforms the semantic parser of Kwiatkowski et al. (2010) when the latter is trained without corpus-based initialization. The authors describe their work as the first model of child acquisition of syntax and semantics to be evaluated by parsing unseen sentences.
  • Fast mapping: the model can learn rare words, in particular the quantifiers a, another and any, after a single example, especially at later stages of learning.
  • Word order: the model learns the correct SVO order even when each utterance has 1, 3, 5 or 7 candidate meanings. Greater ambiguity slows learning, but after an initial period of uncertainty, convergence is abrupt. This refutes the objection of Thornton and Tesan (2007) that statistical models learn too gradually.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics: Psycholinguistics & language acquisition

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kwiatkowski2012probabilistic,
  title = {{A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings}},
  author = {Kwiatkowski, Tom and Goldwater, Sharon and Zettlemoyer, Luke and Steedman, Mark},
  booktitle = {Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {2012},
  url = {https://aclanthology.org/E12-1024/}
}

References

45 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alishahi and Stevenson, S. (2008). A computational model for early argument structure acquisition. Cognitive Science, 32:5:789–834. find
  2. Alishahi, A. and Stevenson, S. (2010). Learning general properties of semantic roles from usage data: a computational model. Language and Cognitive Processes, 25:1. find
  3. Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. Technical report, Gatsby Institute, UCL. find
  4. Börschinger, B., Jones, B. K., and Johnson, M. (2011). Reducing grounded learning tasks to grammatical inference. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1416– 1425, Edinburgh, Scotland, UK. Association for Computational Linguistics. find
  5. Brown, R. (1973). A First Language: the Early Stages. Harvard University Press, Cambridge MA. find
  6. Buttery, P. J. (2006). Computational models for first language acquisition. Technical Report UCAM-CL-TR-675, University of Cambridge, Computer Laboratory. find
  7. Carey, S. and Bartlett, E. (1978). Acquring a single new word. Papers and Reports on Child Language Development, 15. find
  8. Chen, D. L., Kim, J., and Mooney, R. J. (2010). Training a multilingual sportscaster: Using perceptual context to learn language. J. Artif. Intell. Res. (JAIR), 37:397–435. find
  9. Fazly, A., Alishahi, A., and Stevenson, S. (2010). A probabilistic computational model of cross-situational word learning. Cognitive Science, 34(6):1017–1063. find
  10. Frank, M., Goodman, S., and Tenenbaum, J. (2009). Using speakers referential intentions to model early cross-situational word learning. Psychological Science, 20(5):578–585. find
  11. Frank, M. C., Goodman, N. D., and Tenenbaum, J. B. (2008). A bayesian framework for cross-situational word-learning. Advances in Neural Information Processing Systems 20. find
  12. Gibson, E. and Wexler, K. (1994). Triggers. Linguistic Inquiry, 25:355–407. find
  13. Gleitman, L. (1990). The structural sources of verb meanings. Language Acquisition, 1:1–55. find
  14. Goldwater, S., Griffiths, T. L., and Johnson, M. (2009). A Bayesian framework for word segmentation: Exploring the effects of context. Cognition, 112(1):21–54. find
  15. Griffiths, T. L., . T. J. B. (2005). Structure and strength in causal induction. Cognitive Psychology, 51:354–384. find
  16. Griffiths, T. L. and Tenenbaum, J. B. (2006). Optimal predictions in everyday cognition. Psychological Science. find
  17. Hoffman, M., Blei, D. M., and Bach, F. (2010). Online learning for latent dirichlet allocation. In NIPS. find
  18. Kate, R. J. and Mooney, R. J. (2007). Learning language semantics from ambiguous supervision. In Proceedings of the 22nd Conference on Artificial Intelligence (AAAI-07). find
  19. Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., and Steedman, M. (2010). Inducing probabilistic CCG grammars from logical form with higher-order unification. In Proceedings of the Conference on Emperical Methods in Natural Language Processing. find
  20. Kwiatkowski, T., Zettlemoyer, L., Goldwater, S., and Steedman, M. (2011). Lexical generalization in ccg grammar induction for semantic parsing. In Proceedings of the Conference on Emperical Methods in Natural Language Processing. find our card
  21. Levy, R., Reali, F., and Griffiths, T. (2009). Modeling the effects of memory on human online sentence processing with particle filters. In Advances in Neural Information Processing Systems 21. find
  22. Lu, W., Ng, H. T., Lee, W. S., and Zettlemoyer, L. S. (2008). A generative model for parsing natural language to meaning representations. In Proceedings of The Conference on Empirical Methods in Natural Language Processing. find
  23. MacWhinney, B. (2000). The CHILDES project: tools for analyzing talk. Lawrence Erlbaum, Mahwah, NJ u.a. EN. find
  24. Maurits, L., Perfors, A., and Navarro, D. (2009). Joint acquisition of word order and word reference. In Proceedings of the 31th Annual Conference of the Cognitive Science Society. find
  25. Pearl, L., Goldwater, S., and Steyvers, M. (2010). How ideal are we? Incorporating human limitations into Bayesian models of word segmentation. pages 315–326, Somerville, MA. Cascadilla Press. find
  26. Perfors, A., Tenenbaum, J. B., and Regier, T. (2011). The learnability of abstract syntactic principles. Cognition, 118(3):306 – 338. find
  27. Sagae, K., MacWhinney, B., and Lavie, A. (2004). Adding syntactic annotations to transcripts of parent-child dialogs. In Proceedings of the 4th International Conference on Language Resources and Evaluation. Lisbon, LREC. find
  28. Sakas, W. and Fodor, J. D. (2001). The structural triggers learner. In Bertolo, S., editor, Language Acquisition and Learnability, pages 172–233. Cambridge University Press, Cambridge. find
  29. Sanborn, A. N., Griffiths, T. L., and Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review. find
  30. Sato, M. (2001). Online model selection based on the variational bayes. Neural Computation, 13(7):1649–1681. find
  31. Shi, L., Griffiths, T. L., Feldman, N. H., and Sanborn, A. N. (2010). Exemplar models as a mechanism for performing bayesian inference. Psychonomic Bulletin & Review, 17(4):443– 464. find
  32. Siskind, J. M. (1992). Naive Physics, Event Perception, Lexical Semantics, and Language Acquisition. PhD thesis, Massachusetts Institute of Technology. find
  33. Siskind, J. M. (1996). A computational study of cross-situational techniques for learning word-to-meaning mappings. Cognition, 61(1-2):1– 38. find
  34. Steedman, M. (2000). The Syntactic Process. MIT Press, Cambridge, MA. find
  35. Steedman, M. (2002). Plans, affordances, and combinatory grammar. Linguistics and Philosophy, 25. find
  36. Thornton, R. and Tesan, G. (2007). Categorical acquisition: Parameter setting in universal grammar. Biolinguistics, 1. find
  37. Villavicencio, A. (2002). The acquisition of a unification-based generalised categorial grammar. Technical Report UCAM-CL-TR-533, University of Cambridge, Computer Laboratory. find
  38. Wong, Y. W. and Mooney, R. (2006). Learning for semantic parsing with statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL. find our card
  39. Wong, Y. W. and Mooney, R. (2007). Learning synchronous grammars for semantic parsing with lambda calculus. In Proceedings of the Association for Computational Linguistics. find our card
  40. Xu, F. and Tenenbaum, J. B. (2007). Word learning as Bayesian inference. Psychological Review, 114:245–272. find
  41. Yang, C. (2002). Knowledge and Learning in Natural Language. Oxford University Press, Oxford. find
  42. Yu, C. and Ballard, D. H. (2007). A unified model of early word learning: Integrating statistical and social cues. Neurocomputing, 70(13-15):2149 – 2165. find
  43. Zettlemoyer, L. S. and Collins, M. (2005). Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Conference on Uncertainty in Artificial Intelligence. find our card
  44. Zettlemoyer, L. S. and Collins, M. (2007). Online learning of relaxed CCG grammars for parsing to logical form. In Proc. of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. find
  45. Zettlemoyer, L. S. and Collins, M. (2009). Learning context-dependent mappings from sentences to logical form. In Proceedings of The Joint Conference of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kwiatkowski-2012-probabilistic-model-syntactic-semantic-acquisition/