Elamur Research⌕
☰
home / papers / 2025 / Do Construction Distributions Shape Formal Language Learning In German BabyLMs?

Do Construction Distributions Shape Formal Language Learning In German BabyLMs?

Paper · 2025 · Bastian Bunzeck, Daniel Duran and Sina Zarrieß

Summary

Child-directed speech (CDS) is dominated by questions, imperatives and fragments, with few canonical or complex sentences. It is an open question whether such input helps or hurts the learning of formal grammar. The authors test this with small language models for German. They assemble a new developmentally plausible German BabyLM corpus from sources such as CHILDES, OpenSubtitles, Project Gutenberg and MiniKlexikon. They then carry out what they describe as the first utterance-level construction analysis for German, and the resulting profile resembles those reported for English and other languages. From this corpus they draw three 5M-token subsets with different construction mixes and train small Llama models on each, using both character-level and subword tokenization. The models are probed with lexical, syntactic and semantic minimal pairs. Construction distribution turns out to matter little:

  • Complex input gives slightly better syntax.
  • Fragment-rich input gives slightly better word-level knowledge.
  • Complexity changes how steep the learning curves are, but not how much data is needed before learning starts.

This suggests the amount of input matters more than its formal complexity.

Key points

  • The three training subsets are size-matched at 5M tokens and differ mainly in their share of fragmentary versus complex utterances; one of them, "cds", follows a developmentally plausible profile.
  • Models are small character-based and subword Llama variants. Evaluation uses minimal-pair suites, including CLAMS for subject–verb agreement.
  • Despite many German yes/no-questions with inverted subject and verb, the cds subset did not hurt CLAMS agreement performance. It also reached a better end state on word-level tasks.
  • The cds subset has roughly 820,000 sentences. At an estimated 7,000 utterances heard per day, this corresponds to about 120 days of a child's input.
  • Synthetic child-style corpora (TinyStories, TinyDialogues) contain far fewer fragments, questions and short SV(X) utterances than real data. German-specific syntactic evaluation sets are scarce, with MultiBLiMP only recently emerging.

Interesting ideas

  • I1. Hold training-set size fixed and vary only the construction mix, so that any score difference can be credited to composition rather than volume.
  • I2. Profile a corpus by counting utterance-level construction types (questions, imperatives, fragments, copula, SV(X), complex sentences) as a cheap fingerprint of register.
  • I3. Report learning trajectories across checkpoints and separate when learning starts from how fast it improves, instead of comparing only final accuracy.
  • I4. Before trusting generated data, check its construction statistics against authentic text, because LLM output tends to be overly well-formed.
  • I5. Probe lexical, syntactic and semantic knowledge with separate minimal-pair suites, since one data choice can help one level while slightly hurting another.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Build equal-sized gparse training sets with different proportions of fragments, questions and complex clauses, to find which mix actually improves attachment accuracy.
  • I2 (frames): Compare the speech-act distribution of an input text with known registers, so that prag can expect, for example, many imperatives in instructions or many questions in dialogue.
  • I3 (learn): Log per-checkpoint scores for parser and tagger runs and flag data changes that only alter the slope versus those that delay the start.
  • I4 (learn): Measure the fragment and question rates of LLM-teacher silver data against real corpora, and reweight or reject batches whose profile is too clean.
  • I5 (learn): Keep distinct negative-control minimal pairs per pipeline stage, so that a training change that improves the tree but degrades lemmas or frames is caught.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bielefeld UniversityBielefeld · Germany35 works · 1,196 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksDevelopmental and Educational Psychology · 523 works

Cite

Cite as Bastian Bunzeck, Daniel Duran and Sina Zarrieß. 2025. Do Construction Distributions Shape Formal Language Learning In German BabyLMs?. Proceedings of the 29th Conference on Computational Natural Language Learning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.conll-1.12

@inproceedings{bunzeck2025do,
  title = {{Do Construction Distributions Shape Formal Language Learning In German BabyLMs?}},
  author = {Bastian Bunzeck, Daniel Duran and Sina Zarrieß},
  booktitle = {Proceedings of the 29th Conference on Computational Natural Language Learning},
  year = {2025},
  doi = {10.18653/v1/2025.conll-1.12},
  url = {https://doi.org/10.18653/v1/2025.conll-1.12}
}

References

111 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Haifa Alroqi, Ludovica Serratrice, and Thea Cameron-Faulkner. 2023. The association between screen media quantity, content, and context and language development. Journal of Child Language, 50(5):1155– 1183. ↗
  2. Ben Ambridge, Evan Kidd, Caroline F. Rowland, and Anna L. Theakston. 2015. The ubiquity of frequency effects in first language acquisition. Journal of Child Language, 42(2):239–273. ↗
  3. Ben Ambridge and Elena Lieven. 2011. Child Language Acquisition: Contrasting Theoretical Approaches. Cambridge University Press, Cambridge ; New York. ↗
  4. Heike Behrens. 2021. Constructivist Approaches to First Language Acquisition. Journal of Child Language, 48(5):959–983. ↗
  5. Elika Bergelson, Melanie Soderstrom, Iris-Corinna Schwarz, Caroline F. Rowland, Nairán Ramírez-Esparza, Lisa R. Hamrick, Ellen Marklund, Marina Kalashnikova, Ava Guez, Marisa Casillas, Lucia Benetti, Petra Van Alphen, and Alejandrina Cristia. 2023. Everyday language input and production in 1,001 children from six continents. Proceedings of the National Academy of Sciences, 120(52):e2300671120. ↗
  6. Ruth A Berman and Dan I. Slobin. 1994. Different Ways of Relating Events in Narrative: A Crosslinguistic Study. Erlbaum Associates, Hillsdale, NJ. find
  7. Raj Sanjay Shah, Alex Warstadt, Ethan Wilcox, and Adina Williams. 2025. BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop. arXiv preprint. find
  8. Robert C. Berwick, Paul Pietroski, Beracah Yankama, and Noam Chomsky. 2011. Poverty of the Stimulus Revisited. Cognitive Science, 35(7):1207–1242. ↗
  9. Jidong Chen and Yasuhiro Shirai. 2015. The acquisition of relative clauses in spontaneous child speech in Mandarin Chinese. Journal of Child Language, 42(2):394–422. ↗
  10. Silke Brandt, Holger Diessel, and Michael Tomasello. 2008. The acquisition of German relative clauses: A case study*. Journal of Child Language, 35(2):325– 348. ↗
  11. John Bunce, Melanie Soderstrom, Elika Bergelson, Celia Rosemberg, Alejandra Stein, Florencia Alam, Maia Julieta Migdalek, and Marisa Casillas. 2024. A cross-linguistic examination of young children’s everyday language experiences. Journal of Child Language, pages 1–29. ↗
  12. Bastian Bunzeck and Holger Diessel. 2024. The richness of the stimulus: Constructional variation and development in child-directed speech. First Language, 45(2):152–176. ↗
  13. Bastian Bunzeck, Daniel Duran, Leonie Schade, and Sina Zarrieß. 2025. Small language models also work with small vocabularies: Probing the linguistic abilities of grapheme- and phoneme-based baby llamas. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6039– 6048, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  14. Bastian Bunzeck and Sina Zarrieß. 2024. Fifty shapes of BLiMP: Syntactic learning curves in language models are not uniform, but sometimes unruly. In Proceedings of the 2024 CLASP Conference on Multimodality and Interaction in Language Learning, pages 39–55, Gothenburg, Sweden. Association for Computational Linguistics. ↗
  15. Bastian Bunzeck and Sina Zarrieß. 2025. Subword models struggle with word learning, but surprisal hides it. arXiv preprint. ↗
  16. Noam Chomsky. 1965. Aspects of the Theory of Syntax. Number 11 in Massachusetts Institute of Technology. Research Laboratory of Electronics. Special Technical Report. MIT Press, Cambridge, Massachusetts. ↗
  17. Noam Chomsky. 1980. Rules and representations. Behavioral and Brain Sciences, 3(1):1–15. ↗
  18. Leshem Choshen, Ryan Cotterell, Michael Y. Hu, Tal Linzen, Aaron Mueller, Candace Ross, Alex Warstadt, Ethan Wilcox, Adina Williams, and Chengxu Zhuang. 2024. [Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus. Preprint, arXiv:2404.06214. ↗
  19. Morten H Christiansen and Nick Chater. 1999. Toward a Connectionist Model of Recursion in Human Linguistic Performance. Cognitive Science, 23(2):157–205. ↗
  20. Alexander Clark and Shalom Lappin. 2011. Linguistic Nativism and the Poverty of the Stimulus, 1 edition. Wiley. find
  21. Stephen Crain and Paul Pietroski. 2001. Nature, Nurture And Universal Grammar. Linguistics and Philosophy, 24(2):139–186. ↗
  22. Alejandrina Cristia, Emmanuel Dupoux, Nan Bernstein Ratner, and Melanie Soderstrom. 2019. Segmentability Differences Between Child-Directed and Adult-Directed Speech: A Systematic Test With an Ecologically Valid Corpus. Open Mind, 3:13–22. ↗
  23. Holger Diessel. 2019. The Grammar Network. Cambridge University Press, Cambridge. ↗
  24. Thea Cameron-Faulkner and Tina Hickey. 2011. Form and function in Irish child directed speech. Cognitive Linguistics, 22(3):569–594. ↗
  25. Holger Diessel. 2023. The Constructicon: Taxonomies and Networks, 1 edition. Cambridge University Press. find
  26. Thea Cameron-Faulkner, Elena Lieven, and Michael Tomasello. 2003. A construction based analysis of child directed speech. Cognitive Science, 27(6):843– 873. ↗
  27. Holger Diessel and Michael Tomasello. 2000. The development of relative clauses in spontaneous child speech. Cognitive Linguistics, 11(1-2):131–151. ↗
  28. Thea Cameron-Faulkner and Claire Noble. 2013. A comparison of book text and Child Directed Speech. First Language, 33(3):268–279. ↗
  29. G. William Domhoff and Adam Schneider. 2008. Studying dream content using the archive and search engine on DreamBank.net. Consciousness and Cognition, 17(4):1238–1247. ↗
  30. Franklin Chang, Gary S. Dell, and Kathryn Bock. 2006. Becoming syntactic. Psychological Review, 113(2):234–272. find
  31. Lucas Charpentier, Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Michael Hu, Jaap Jumelet, Tal Linzen, Jing Liu, Aaron Mueller, Candace Ross, Lukas Edman, Lisa Bylinina, Faeze Ghorbanpour, and Alexander Fraser. 2024. Are BabyLMs second language learners? In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 166–173, Miami, FL, USA. Association for Computational Linguistics. find
  32. Ronen Eldan and Yuanzhi Li. 2023. TinyStories: How Small Can Language Models Be and Still Speak Coherent English? Preprint, arXiv:2305.07759. ↗
  33. Jeffrey L. Elman. 1993. Learning and development in neural networks: The importance of starting small. Cognition, 48(1):71–99. ↗
  34. Steven Y. Feng, Noah Goodman, and Michael Frank. 2024. Is child-directed speech effective training data for language models? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22055–22071, Miami, Florida, USA. Association for Computational Linguistics.
  35. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. find
  36. Giuliana Genovese, Maria Spinelli, Leonor J. Romero Lauro, Tiziana Aureli, Giulia Castelletti, and Mirco Fasolo. 2020. Infant-directed speech as a simplified but not simple register: A longitudinal study of lexical and syntactic features. Journal of Child Language, 47(1):22–44. ↗
  37. Martin Gerlach and Francesc Font-Clos. 2020. A Standardized Project Gutenberg Corpus for Statistical Analysis of Natural Language and Quantitative Linguistics. Entropy, 22(1):126. ↗
  38. Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC‘12), pages 759–765, Istanbul, Turkey. European Language Resources Association (ELRA).
  39. Zébulon Goriely and Paula Buttery. 2025a. BabyLM’s First Words: Word Segmentation as a Phonological Probing Task. arXiv preprint. find
  40. Zébulon Goriely and Paula Buttery. 2025b. IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modeling. arXiv preprint.
  41. Zébulon Goriely, Richard Diehl Martinez, Andrew Caines, Paula Buttery, and Lisa Beinborn. 2024. From babble to words: Pre-training language models on continuous streams of phonemes. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 37–53, Miami, FL, USA. Association for Computational Linguistics. ↗
  42. Anna Elizabeth Gowenlock, Courtenay Norbury, and Jennifer M. Rodd. 2024. Exposure to Language in Video and its Impact on Linguistic Development in Children Aged 3–11: A Scoping Review. Journal of Cognition, 7(1):57. ↗
  43. Maria Teresa Guasti. 2002. Language Acquisition: The Growth of Grammar. MIT Press, Cambridge, Mass. find
  44. John Hale. 2001. A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics.
  45. Linyang He, Ercong Nie, Sukru Samet Dindar, Arsalan Firoozi, Adrian Florea, Van Nguyen, Corentin Puffay, Riki Shimizu, Haotian Ye, Jonathan Brennan, Helmut Schmid, Hinrich Schütze, and Nima Mesgarani. 2025. XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs. arXiv preprint. ↗
  46. Joseph Henrich. 2024. WEIRD. In Open Encyclopedia of Cognitive Science, 1 edition. MIT Press. find
  47. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015. The Goldilocks Principle: Reading Children’s Books with Explicit Memory Representations. Preprint, arXiv:1511.02301. ↗
  48. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength natural language processing in python. find
  49. Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen, Chengxu Zhuang, Ryan Cotterell, Leshem Choshen, Alex Warstadt, and Ethan Gotlieb Wilcox. 2024. Findings of the second BabyLM challenge: Sample-efficient pretraining on developmentally plausible corpora. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 1–21, Miami, FL, USA. Association for Computational Linguistics. ↗
  50. Philip A. Huebner, Elior Sulem, Fisher Cynthia, and Dan Roth. 2021. BabyBERTa: Learning More Grammar With Small-Scale Child-Directed Language. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 624–646, Online. Association for Computational Linguistics.
  51. Janellen Huttenlocher, Wendy Haight, Anthony Bryk, Michael Seltzer, and Thomas Lyons. 1991. Early vocabulary growth: Relation to language input and gender. Developmental Psychology, 27(2):236–248. ↗
  52. Janellen Huttenlocher, Marina Vasilyeva, Elina Cymer-man, and Susan Levine. 2002. Language input and child syntax. Cognitive Psychology, 45(3):337–374. ↗
  53. Daniel Jach and Gunther Dietz. 2024. KORPUS EIN-FACHES DEUTSCH (KED). Korpora Deutsch als Fremdsprache, 4. find
  54. Laura A. Janda, editor. 2013. Cognitive Linguistics: The Quantitative Turn: The Essential Reader. Mouton Reader. De Gruyter Mouton, Berlin. find
  55. Jaap Jumelet, Leonie Weissweiler, and Arianna Bisazza. 2025. MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs. arXiv preprint. ↗
  56. Krisjanis Karins, Robert MacIntyre, Monika Brandmair, Susanne Lauscher, and Cynthia McLemore. 1997. CALLHOME German Transcripts. find
  57. Emmanuel Keuleers and Marc Brysbaert. 2010. Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42(3):627–633. ↗
  58. Marc Kupietz, Cyril Belica, Holger Keibel, and Andreas Witt. 2010. The German reference corpus DeReKo: A primordial sample for linguistic research. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC‘10), Valletta, Malta. European Language Resources Association (ELRA). find
  59. Catherine Laing and Elika Bergelson. 2024. Analyzing the effect of sibling number on input and output in the first 18 months. Infancy, 29(2):175–195. ↗
  60. Marvin Lavechin, Maureen De Seyssel, Hadrien Titeux, Guillaume Wisniewski, Hervé Bredin, Alejandrina Cristia, and Emmanuel Dupoux. 2025. Simulating Early Phonetic and Word Learning Without Linguistic Categories. Developmental Science, 28(2):e13606. ↗
  61. Marvin Lavechin, Yaya Sy, Hadrien Titeux, María Andrea Cruz Blandón, Okko Räsänen, Hervé Bredin, Emmanuel Dupoux, and Alejandrina Cristia. 2023. BabySLM: Language-acquisition-friendly benchmark of self-supervised spoken language models. In INTERSPEECH 2023, pages 4588–4592. ISCA. find
  62. Gaël Le Godais, Tal Linzen, and Emmanuel Dupoux. 2017. Comparing character-level neural language models using a lexical decision task. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 125–130, Valencia, Spain. Association for Computational Linguistics. find
  63. Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 923–929, Portorož, Slovenia. European Language Resources Association (ELRA).
  64. Bria Long, Violet Xiang, Stefan Stojanov, Robert Z. Sparks, Zi Yin, Grace E. Keene, Alvin W. M. Tan, Steven Y. Feng, Chengxu Zhuang, Virginia A. Marchman, Daniel L. K. Yamins, and Michael C. Frank. 2024. The BabyView dataset: High-resolution egocentric videos of infants’ and young children’s everyday experiences. arXiv preprint. find
  65. Brian MacWhinney. 2000. The CHILDES Project: Tools for Analyzing Talk, 3 edition. Lawrence Erlbaum Associates, Mahwah, NJ. find
  66. Brian MacWhinney. 2004. A multiple process solution to the logical problem of language acquisition. Journal of Child Language, 31(4):883–914. ↗
  67. Kyle Mahowald, Anna A. Ivanova, Idan A. Blank, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2024. Dissociating language and thought in large language models. Trends in Cognitive Sciences, pages 517–540. ↗
  68. Raphaël Millière. 2024. Language Models as Models of Language. arXiv preprint.
  69. Kanishka Misra. 2022. Minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models. arXiv preprint. ↗
  70. Kanishka Misra and Kyle Mahowald. 2024. Language models learn rare phenomena from less rare phenomena: The case of the missing AANNs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 913–929, Miami, Florida, USA. Association for Computational Linguistics. ↗
  71. Jessica L. Montag. 2019. Differences in sentence complexity in the text of children’s picture books and child-directed speech. First Language, 39(5):527– 546. ↗
  72. Sherin Muckatira, Vijeta Deshpande, Vladislav Lialin, and Anna Rumshisky. 2024. Emergent Abilities in Reduced-Scale Generative Language Models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1242–1257, Mexico City, Mexico. Association for Computational Linguistics. find
  73. Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, Natalia Talmina, and Tal Linzen. 2020. Cross-Linguistic Syntactic Evaluation of Word Prediction Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5523–5539, Online. Association for Computational Linguistics.
  74. Claire Noble, Thea Cameron-Faulkner, Andrew Jessop, Anna Coates, Hannah Sawyer, Rachel Taylor-Ims, and Caroline F. Rowland. 2020. The Impact of Interactive Shared Book Reading on Children’s Language Skills: A Randomized Controlled Trial. Journal of Speech, Language, and Hearing Research, 63(6):1878–1897. ↗
  75. Claire H. Noble, Thea Cameron-Faulkner, and Elena Lieven. 2018. Keeping it simple: The grammatical properties of shared book reading. Journal of Child Language, 45(3):753–766. ↗
  76. Francesca Padovani, Jaap Jumelet, Yevgen Matusevych, and Arianna Bisazza. 2025. Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models. arXiv preprint.
  77. Ludovica Pannitto and Aurélie Herbelot. 2020. Recurrent babbling: Evaluating the acquisition of grammar from limited input data. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 165–176, Online. Association for Computational Linguistics. ↗
  78. Abhinav Patil, Jaap Jumelet, Yu Ying Chiu, Andy Lapastora, Peter Shen, Lexie Wang, Clevis Willrich, and Shane Steinert-Threlkeld. 2024. Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence. Transactions of the Association for Computational Linguistics, 12:1597–1615. ↗
  79. Steven T. Piantadosi. 2024. Modern language models refute Chomsky’s approach to language. In Edward Gibson and Moshe Poliak, editors, From Fieldwork to Linguistic Theory: A Tribute to Dan Everett, Empirically Oriented Theoretical Morphology and Syntax. Language Science Press, Berlin. ↗
  80. Catherine E. Snow and Charles A. Ferguson, editors. 1977. Talking to Children: Language Input and Acquisition. Cambridge University Press, Cambridge, MA. find
  81. Melanie Soderstrom. 2007. Beyond babytalk: Re-evaluating the nature and content of speech input to preverbal infants. Developmental Review, 27(4):501– 532. ↗
  82. Julian M. Pine. 1994. The language of primary caregivers. In Clare Gallaway and Brian J. Richards, editors, Input and Interaction in Language Acquisition, 1 edition, pages 15–37. Cambridge University Press. ↗
  83. James W. Stigler, Ronald Gallimore, and James Hiebert. 2000. Using Video Surveys to Compare Classrooms and Teaching Across Cultures: Examples and Lessons From the TIMSS Video Studies. Educational Psychologist, 35(2):87–100. ↗
  84. Laurent Prévot, Sheng-Fu Wang, Jou-An Chi, and Shu-Kai Hsieh. 2024. Extending the BabyLM initiative : Promoting diversity in datasets and metrics through high-quality linguistic corpora. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 147–158, Miami, FL, USA. Association for Computational Linguistics. find
  85. Alice Suozzi, Luca Capone, Gianluca E. Lebani, and Alessandro Lenci. 2025. BAMBI: Developing Baby Language Models for Italian. arXiv preprint. ↗
  86. Geoffrey K Pullum and Barbara C Scholz. 2002. Empirical assessment of stimulus poverty arguments. The Linguistic Review, 18(1-2). ↗
  87. Silke Reineke, Arnulf Deppermann, and Thomas Schmidt. 2023. Das Forschungs- und Lehrkorpus für Gesprochenes Deutsch (FOLK). In Arnulf Deppermann, Christian Fandrych, Marc Kupietz, and Thomas Schmidt, editors, Korpora in Der Germanistischen Sprachwissenschaft, pages 71–102. De Gruyter. ↗
  88. Meredith L. Rowe. 2012. A Longitudinal Investigation of the Role of Quantity and Quality of Child-Directed Speech in Vocabulary Development. Child Development, 83(5):1762–1774. ↗
  89. Joshua Rozner, Leonie Weissweiler, Kyle Mahowald, and Cory Shain. 2025a. Constructions are Revealed in Word Distributions. arXiv preprint. ↗
  90. Joshua Rozner, Leonie Weissweiler, and Cory Shain. 2025b. BabyLM’s First Constructions: Causal interventions provide a signal of learning. arXiv preprint. find
  91. David E. Rumelhart and James L. McClelland. 1986. On Learning the Past Tenses of English Verbs. In Parallel Distributed Processing, volume 2, pages 535– 551. MIT Press, Cambridge, MA. find
  92. Suchir Salhan, Richard Diehl Martinez, Zébulon Goriely, and Paula Buttery. 2024. Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies. arXiv preprint. ↗
  93. Nazanin Shafiabadi and Guillaume Wisniewski. 2025. Beyond surprisal: A dual metric framework for lexical skill acquisition in LLMs. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6636–6641, Abu Dhabi, UAE. Association for Computational Linguistics. find
  94. Shira Tal, Eitan Grossman, and Inbal Arnon. 2024. Infant-directed speech becomes less redundant as infants grow: Implications for language learning. Cognition, 249:105817. ↗
  95. Margaret Thomas. 2002. Development of the concept of “the poverty of the stimulus”. The Linguistic Review, 18(1-2). find
  96. Michael Tomasello. 2005. Beyond formalities: The case of language acquisition. The Linguistic Review, 22(2-4). ↗
  97. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. Preprint, arXiv:2302.13971. ↗
  98. Mark VanDam, Anne Warlaumont, Elika Bergelson, Alejandrina Cristia, Melanie Soderstrom, Paul De Palma, and Brian MacWhinney. 2016. Home-Bank: An Online Repository of Daylong Child-Centered Audio Recordings. Seminars in Speech and Language, 37(02):128–142. find
  99. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  100. Tom Viering and Marco Loog. 2023. The Shape of Learning Curves: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7799–7819. ↗
  101. Wai Keen Vong, Wentao Wang, A. Emin Orhan, and Brenden M. Lake. 2024. Grounded language acquisition through the eyes and ears of a single child. Science, 383(6682):504–511. ↗
  102. Wentao Wang, Wai Keen Vong, Najoung Kim, and Brenden M. Lake. 2023. Finding Structure in One Child’s Linguistic Experience. Cognitive Science, 47(6):e13305. ↗
  103. Renate Zangl and Debra L. Mills. 2007. Increased Brain Activity to Infant-Directed Speech in 6- and 13-Month-Old Infants. Infancy, 11(1):31–62. ↗
  104. Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Gotlieb Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, and Ryan Cotterell. 2023. Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning, pages 1–6, Singapore. Association for Computational Linguistics. find
  105. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The Benchmark of Linguistic Minimal Pairs for English. Transactions of the Association for Computational Linguistics, 8:377– 392. ↗
  106. Leonie Weissweiler, Taiqi He, Naoki Otani, David R. Mortensen, Lori Levin, and Hinrich Schütze. 2023. Construction Grammar Provides Unique Insight into Neural Language Models. In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), pages 85–95. ↗
  107. Leonie Weissweiler, Kyle Mahowald, and Adele Goldberg. 2025. Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs. arXiv preprint. ↗
  108. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick Von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  109. Hong Meng Yam and Nathan Paek. 2024. What should baby models read? Exploring sample-efficient data composition on model performance. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 284– 291, Miami, FL, USA. Association for Computational Linguistics. find
  110. Charles D. Yang. 2004. Universal Grammar, statistics or both? Trends in Cognitive Sciences, 8(10):451– 456. ↗
  111. Daniel Yurovsky, Chen Yu, and Linda B. Smith. 2012. Statistical Speech Segmentation and Word Learning in Parallel: Scaffolding from Child-Directed Speech. Frontiers in Psychology, 3. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bunzeck-2025-do-construction-distributions-shape-formal/