Elamur Research⌕
☰
home / papers / 2024 / A systematic investigation of learnability from single child linguistic input

A systematic investigation of learnability from single child linguistic input

Paper · 2024 · Yulu Qin, Wentao Wang and Brenden M. Lake

Summary

Modern language models learn from vastly more text than a child hears, and that text is very different from speech addressed to children. This makes it hard to use them as evidence about how humans learn language. Earlier work by Wang et al. (2023) trained CBOW and LSTM models on a single child's input (SAYCam-S). The authors test whether those results hold more broadly. They train six architectures on five corpora: LSTMs, GPT-2-style Transformers and BabyBERTa-style Transformers, each in two sizes. Three corpora are single-child datasets (SAYCam-S, Sarah, Ellie), and two are baselines: a size-matched Wikipedia sample and the aggregated North American CHILDES. Each model is evaluated three ways: a regenerated full Zorro grammaticality suite, word-embedding visualizations, and noun/verb cloze tests. Models trained on single-child data behave consistently across architectures. They form meaningful syntactic and semantic word categories and handle several grammatical phenomena. They still fail on harder constructions, such as agreement across relative clauses. These findings strengthen the claim that a single child's input, roughly 200K tokens, can support useful distributional learning.

Key points

  • Datasets:
    • SAYCam-S has about 212K training tokens.
    • Sarah comes from the Brown corpus and has about 221K tokens.
    • Ellie comes from the Sakali corpus and has about 240K tokens.
    • A Wikipedia sample with about 261K tokens serves as a baseline.
    • North American CHILDES serves as a reference, with about 8.2M tokens, around 30× the single-child sets.
  • Models: LSTM (1 and 2 layers), GPT-2 (2 and 8 layers) and BabyBERTa (2 and 8 layers), ranging from 3.3M to 26.8M parameters. All are trained from scratch, with hyperparameters tuned on validation perplexity and results averaged over 3 seeds.
  • Zorro suite: Zorro was rebuilt from the intersected vocabulary of the five datasets. This gives all 23 tests covering 13 phenomena, with 2,000 pairs each. Wang et al. had used only 15 filtered tests.
  • Zorro accuracy by training data:
    • Single-child data: roughly 64–70%.
    • Wikipedia: about 57–62%.
    • CHILDES: about 78–88%.
  • Noun/verb cloze: every model exceeded 90% accuracy on the noun-vs-verb cloze tests. For comparison, the smallest BabyLM track uses about 40× more data than one single-child dataset.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓New York UniversityNew York · United States201 works · 100,180 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Yulu Qin, Wentao Wang and Brenden M. Lake. 2024. A systematic investigation of learnability from single child linguistic input. arXiv:2402.07899.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.07899 · arXiv 2402.07899

@misc{qin2024systematic,
  title = {{A systematic investigation of learnability from single child linguistic input}},
  author = {Yulu Qin, Wentao Wang and Brenden M. Lake},
  year = {2024},
  doi = {10.48550/arxiv.2402.07899},
  eprint = {2402.07899},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.07899}
}

References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abend, O., Kwiatkowski, T., Smith, N. J., Goldwater, S., & Steedman, M. (2017). Bootstrapping language acquisition. Cognition, 164, 116–143. find
  2. Beaupoil-Hourdel, P. (2015). Multimodal acquisition and expression of negation. analysis of a videotaped and longitudinal corpus of a french and an english mother-child dyad (Unpublished doctoral dissertation). find
  3. Ph. D. Dissertation, Sorbonne Nouvelle University, Paris. Brown, R. (1973). A first language: The early stages. find
  4. Harvard University Press. Chang, T. A., & Bergen, B. K. (2023). Language model behavior: A comprehensive survey. arXiv preprint arXiv:2303.11504.
  5. Elman, J. L. (1989). Representation and structure in connectionist models. Center for Research in Language, University of California, San Diego. find
  6. Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211. find
  7. Elman, J. L. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine learning, 7, 195–225. find
  8. Frank, M. C. (2023a). Bridging the data gap between children and large language models. Trends in Cognitive Sciences, 27(11), 990-992. find
  9. Frank, M. C. (2023b). Large language models as models of human cognition. PsyArXiv. Retrieved from https://psyarxiv.com/wxt69 Frank, M. C., Braginsky, M., Yurovsky, D., & Marchman, V. A. (2016). Wordbank: an open repository for developmental vocabulary data*. Journal of Child Language, 44, 677 - 69… ↗
  10. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735–1780. find
  11. Huebner, P. A., Sulem, E., Cynthia, F., & Roth, D. (2021). Babyberta: Learning more grammar with small-scale child-directed language. In Proceedings of the 25th conference on computational natural language learning (pp. 624–646).
  12. Huebner, P. A., & Willits, J. A. (2020). Order matters: Developmentally plausible acquisition of lexical categories. In Cogsci. find
  13. Kim, N., & Smolensky, P. (2021, February). Testing for grammatical category abstraction in neural language models. In Proceedings of the society for computation in linguistics 2021 (pp. 467–470). Online: Association for Computational Linguistics. find
  14. Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse processes, 25(2-3), 259–284. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., . . . Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:19…
  15. MacWhinney, B. (2000). The childes project: Tools for analyzing talk (3rd ed.). find
  16. Lawrence Erlbaum Associates Publishers. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
  17. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9. Sellam, T., Yadlowsky, S., Tenney, I., Wei, J., Saphra, N., D’Amour, A. N., . . . Pavlick, E. (2022). The multiberts: Bert reproductions for… find
  18. Sullivan, J., Mei, M., Perfors, A., Wojcik, E., & Frank, M. C. (2021). Saycam: A large, longitudinal audiovisual dataset recorded from the infant’s perspective. Open mind, 5, 20– 29. find
  19. Taylor, W. L. (1953). “Cloze Procedure”: A New Tool for Measuring Readability. Journalism & Mass Communication Quarterly, 30, 415 - 433. find
  20. Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(11). Vázquez Martı́nez, H., Lea Heuser, A., Yang, C., & Kodner, J. (2023, December). Evaluating neural language models as cognitive models of language acquisition. In Proceedings of the 1s… find
  21. Vong, W. K., Wang, W., Orhan, A. E., & Lake, B. M. (2024). Grounded language acquisition through the eyes and ears of a single child. Science, 383, 504–511. find
  22. Wang, W., Vong, W. K., Kim, N., & Lake, B. M. (2023). Finding structure in one child’s linguistic experience. Cognitive science, 47(6), e13305. find
  23. Warstadt, A., & Bowman, S. R. (2022). What artificial neural networks can tell us about human language acquisition. Algebraic Structures in Natural Language, 17–60. Warstadt, A., Mueller, A., Choshen, L., Wilcox, E., Zhuang, C., Ciro, J., . . . others (2023). Findings of the babylm challenge: Sample…
  24. Waterfall, H. R., Sandbank, B., Onnis, L., & Edelman, S. (2010). An empirical generative framework for computational modeling of language acquisition. Journal of child language, 37(3), 671–703. find
  25. Yedetore, A., Linzen, T., Frank, R., & McCoy, R. T. (2023, July). How poor is the stimulus? evaluating hierarchical generalization in neural networks trained on child-directed speech. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) (…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qin-2024-systematic-investigation-learnability-single-child/