Elamur Research⌕
☰
home / papers / 2023 / Linguistic Productivity: the Case of Determiners in English

Linguistic Productivity: the Case of Determiners in English

Summary

The paper tests whether children need innate syntactic categories in order to freely combine the same noun with different articles (a / the). The alternative hypothesis is that this ability emerges solely from the statistics of the speech children hear. The authors combine corpus analysis with computational modeling. The data come from two corpora of adult–child conversations. The first is Manchester. The second is the much larger Language Development Project (LDP), which also covers younger ages. Productivity is assessed in two ways: with the classic overlap score and with the onset measure, which requires less data. A Transformer model with no predefined categories is trained from scratch on child-directed speech. Its output is then compared with children's output at different developmental stages. At early stages, children do not yet use articles productively; this emerges at around 30 months. The model closely reproduces this trajectory, including the point at which productivity emerges. This ability can therefore be explained by learning from the input without a priori categories. The approach itself is not tied to a specific construction.

Key points

  • Corpora. LDP covers 64 children from the Chicago area aged 14 to 58 months and contains over 1 million transcribed utterances: ~647K from parents and ~369K from children. Manchester covers 12 children aged 20 to 36 months. Parts of speech were tagged with spaCy, and the preprocessing code is available on OSF.
  • Metrics. The overlap score is computed only for samples of at least 50 constructions. The onset measure registers productivity when a child uses both a and the with the same noun within a single session, for at least two different nouns.
  • Result. Children begin to use articles productively only at around 30 months. For the LDP children, the onset of productivity in the model correlates strongly with the onset in the child.
  • The model does not simply memorize. It produces article–noun combinations that did not occur in the training data, for example "a daffodil". Predictions on parents' and children's utterances do not differ significantly.
  • Limitations. The model was trained on pooled data from all caregivers, so individual differences between children are not modeled. The study covers only one category (determiners) and only English.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.ijcnlp-main.21

@inproceedings{alhama2023linguistic,
  title = {{Linguistic Productivity: the Case of Determiners in English}},
  author = {Alhama, Raquel G. and Foushee, Ruthe and Byrne, Daniel and Ettinger, Allyson and Goldin-Meadow, Susan and Alishahi, Afra},
  booktitle = {Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.ijcnlp-main.21},
  url = {https://doi.org/10.18653/v1/2023.ijcnlp-main.21}
}

References

34 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Omri Abend, Tom Kwiatkowski, Nathaniel J Smith, Sharon Goldwater, and Mark Steedman. 2017. Bootstrapping language acquisition. Cognition, 164:116– 143. ↗
  2. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In Proceedings of the International Conference on Learning Representations (ICLR). ↗ our card
  3. Afra Alishahi and Grzegorz Chrupała. 2012. Concurrent acquisition of word meaning and lexical categories. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, pages 643–654. find
  4. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. ↗ our card
  5. Erica A Cartmill, Dea Hunsicker, and Susan Goldin-Meadow. 2014. Pointing and naming are not redundant: children use gesture to modify nouns before they modify nouns in speech. Developmental Psychology, 50(6):1660. ↗
  6. G. Chrupała. 2011. Efficient induction of probabilistic word classes with LDA. In International Joint Conference on Natural Language Processing. find
  7. Grzegorz Chrupała and Afra Alishahi. 2019. Correlating neural and symbolic representations of language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2952– 2962, Florence, Italy. Association for Computational Linguistics. ↗
  8. A. Clark. 2000. Inducing syntactic categories by context distribution clustering. In Proceedings of the 2nd workshop on Learning Language in Logic and the 4th conference on Computational Natural Language Learning, pages 91–94. Association for Computational Linguistics Morristown, NJ, USA. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  10. Stefan L Frank, Padraic Monaghan, and Chara Tsoukala. 2019. Neural network models of language acquisition and processing. In Human language: From genes and brain to behavior, pages 277–293. MIT Press. ↗
  11. Susan Goldin-Meadow, Susan C Levine, Larry V Hedges, Janellen Huttenlocher, Stephen W Rauden-bush, and Steven L Small. 2014. New evidence about language and cognitive development based on a longitudinal study: hypotheses for intervention. American Psychologist, 69(6):588. find
  12. Robert Grimm, Giovanni Cassani, Walter Daelemans, and Steven Gillis. 2015. Towards a model of prediction-based syntactic category acquisition: First steps with word embeddings. In Proceedings of the Sixth Workshop on Cognitive Aspects of Computational Language Learning, pages 28–32, Lisbon, Portugal. Association for Computational Linguistics. find
  13. John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129–4138, Minneapolis, Minnesota. Association for Computational Linguistics. find our card
  14. Matthew Honnibal and Ines Montani. 2017. spacy2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. find
  15. Philip A. Huebner, Elior Sulem, Fisher Cynthia, and Dan Roth. 2021. BabyBERTa: Learning more grammar with small-scale child-directed language. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 624–646, Online. Association for Computational Linguistics. find
  16. Nenagh Kemp, Elena Lieven, and Michael Tomasello. 2005. Young children’s knowledge of the" determiner" and" adjective" categories. Journal of Speech, Language & Hearing Research, 48(3). find
  17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  18. B. MacWhinney. 1995. The CHILDES Project: Tools for Analyzing Talk, second edition. Hillsdale, NJ: Lawrence Erlbaum Associates. find
  19. David Marr. 2010. Vision: A computational investigation into the human representation and processing of visual information. MIT press. find
  20. Stephan C Meylan, Michael C Frank, Brandon C Roy, and Roger Levy. 2017. The emergence of an abstract grammatical category in children’s early speech. Psychological science, 28(2):181–192. ↗
  21. T.H. Mintz. 2003. Frequent frames as a cue for grammatical categories in child directed speech. Cognition, 90(1):91–117. ↗
  22. Ludovica Pannitto and Aurélie Herbelot. 2020. Recurrent babbling: evaluating the acquisition of grammar from limited input data. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 165–176, Online. Association for Computational Linguistics. ↗
  23. Christopher Parisien, Afsaneh Fazly, and Suzanne Stevenson. 2008. An incremental bayesian model for learning syntactic categories. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 89–96. find
  24. Lauren Phillips and Nathan Hodas. 2017. Assessing the linguistic productivity of unsupervised deep neural networks. In Proceedings of the 39th Annual Meeting of the Cognitive Sciencey Society (CogSci 2017), pages 937–942, London, United Kingdom. Austin, Texas:Cognitive Science Society. Demonstrations, pages 38–45, Online. Association for Computational Linguistics. find
  25. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. ↗
  26. Charles Yang. 2011. A statistical test for grammar. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, pages 30–38, Portland, Oregon, USA. Association for Computational Linguistics. find
  27. Charles Yang. 2013. Ontogeny and phylogeny of language. Proceedings of the National Academy of Sciences, 110(16):6324–6327. ↗
  28. Julian M. Pine, Daniel Freudenthal, Grzegorz Krajewski, and Fernand Gobet. 2013. Do young children have adult-like syntactic categories? zipf’s law and the case of the determiner. Cognition, 127(3):345–360. ↗
  29. Julian M Pine and Elena VM Lieven. 1997. Slot and frame patterns and the development of the determiner category. Applied Psycholinguistics, 18(2):123–138. ↗
  30. M. Redington, N. Crater, and S. Finch. 1998. Distributional information: A powerful cue for acquiring syntactic categories. Cognitive Science: A Multidisciplinary Journal, 22(4):425–469. ↗
  31. Harish Tayyar Madabushi, Dagmar Divjak, and Petar Milin. 2022. Abstraction not memory: BERT and the English article system. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 924–931, Seattle, United States. Association for Computational Linguistics. find
  32. Anna L. Theakston, Elena Lieven, Julian M. Pine, and Caroline F. Rowland. 2001. The role of performance limitations in the acquisition of verb-argument structure: an alternative account. Journal of Child Language, 28:127–152. ↗
  33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  34. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alhama-2023-linguistic-productivity-case-determiners-english/