Elamur Research⌕
☰
home / papers / 2026 / Do BabyLMs Wanna Learn Wanna Contraction? On the Learnability without Language-Specific Bias

Do BabyLMs Wanna Learn Wanna Contraction? On the Learnability without Language-Specific Bias

Paper · 2026 · Kangsan Noh and Sanghoun Song

Summary

Wanna contraction is a classic argument for innate grammar. English allows "who do you wanna see" (object extraction) but blocks contraction when the wh-word is extracted from subject position, and children seem to get little direct evidence for this restriction. The authors test whether language models trained on child-sized input can learn it anyway.

They took the embedded verbs that follow wanna in the CHILDES part of the BabyLM 100M training data and built two test sets of 100 minimal pairs each: one with frequent verbs and one with rare verbs. For each pair, they measured the surprisal of the sentence-final word "tomorrow". Encoders predicted it as a masked token and decoders as the next token. A model that knows the constraint should find it more surprising in the subject-extraction version.

They evaluated 24 BabyLMs from the 2024 challenge against four standard models. BERT and GPT-2 were almost perfectly consistent. BabyLMs showed modest sensitivity, and only under certain conditions. The findings suggest the constraint is partly learnable from realistic input without language-specific bias.

Key points

  • Corpus counts: In the BabyLM 100M CHILDES subset, wanna (59,085 occurrences) outnumbers want to (9,057). CHILDES is the only subset where this happens.
  • Verb pairs: The high-frequency set uses go/take and come/see. The low-frequency set uses return/lead and apologize/congratulate. Each pair contributes 50 sentence pairs.
  • Models tested:
    • Four standard baselines: BERT-base-uncased, BERT-large-uncased, GPT-2 and GPT-2-medium.
    • 20 submitted BabyLMs, split evenly between the Strict track (100M words) and the Strict-Small track (10M words).
    • Four challenge baselines: LTG-BERT and BabyLlama.
  • Main pattern: BabyLMs did better when trained on more data and when tested on high-frequency verbs. Only encoder-based BabyLMs captured the constraint, and babylm24_MLSM was the most consistent.
  • Availability: Code and data are released at github.com/San0727/wanna-babylm.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Korea UniversitySeoul · South Korea26 works · 9,490 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Kangsan Noh and Sanghoun Song. 2026. Do BabyLMs Wanna Learn Wanna Contraction? On the Learnability without Language-Specific Bias. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.552

@inproceedings{noh2026do,
  title = {{Do BabyLMs Wanna Learn Wanna Contraction? On the Learnability without Language-Specific Bias}},
  author = {Kangsan Noh and Sanghoun Song},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.552},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.552}
}

References

54 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pablo Contreras Kallens, Ross Deans Kristensen-McLachlan, and Morten H. Christiansen. 2023. Large language models demonstrate the potential of statistical learning in language. Cognitive Science, 47(3):e13256. find
  2. Ben Ambridge and Liam Blything. 2024. Large language models are better than theoretical linguists at theoretical linguistics. Theoretical Linguistics, 50(1– 2):33–48. find
  3. Stephen Crain and Rosalind Thornton. 1998. Investigations in Universal Grammar: A Guide to Experiments on the Acquisition of Syntax and Semantics. MIT Press, Cambridge, MA. find
  4. Rufus Behr. 2024. ELC-parserBERT: Low-resource language modeling utilizing a parser network with ELC-BERT. In Proceedings of the 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 140–146, Miami, Florida. Association for Computational Linguistics. find
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  6. Gábor Berend. 2024. Integrating quasi-symbolic conceptual knowledge into language model pre-training. In Proceedings of the 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 159–165, Miami, Florida. Association for Computational Linguistics. find
  7. Idan A. Blank. 2023. What are large language models supposed to model? Trends in Cognitive Sciences, 27(11):987–989. find
  8. Lukas Edman. 2024. Babylm 2024. https://huggingface.co/collections/leukas/ babylm-2024-66e1bae0dc9eb134560be8ff. Accessed: 2025-09-14. ↗
  9. Hans C. Boas. 2004. You wanna consider a constructional approach towards wanna-contraction? In Michael Achard and Suzanne Kemmer, editors, Language, Culture, and Mind, pages 479–491. CSLI Publications, Stanford, CA. find
  10. Danny Fox and Roni Katzir. 2024. Large language models and theoretical linguistics. Theoretical Linguistics, 50(1–2):71–76. find
  11. Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. Neural language models as psycholinguistic subjects: Representations of syntactic state. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lingui…
  12. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  13. Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy. 2020. SyntaxGym: An online platform for targeted evaluation of language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 70–76, Online. Association for…
  14. Bastian Bunzeck, Daniel Duran, Leonie Schade, and Sina Zarrieß. 2025. Small language models also work with small vocabularies: Probing the linguistic abilities of grapheme- and phoneme-based baby llamas. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6039– 60… find
  15. Heidi R. Getz. 2019. Acquiring Wanna: Beyond universal grammar. Language Acquisition, 26(2):119–143. find
  16. Grant Goodall. 2021. Wanna-contraction as restructuring. In Grant Goodall, editor, Theory and Experiment in Syntax, pages 53–68. Routledge, New York. find
  17. Noam Chomsky. 1977. On Wh-movement. In Peter W. Culicover, Thomas Wasow, and Adrian Akmajian, editors, Formal Syntax, pages 71–132. Academic Press, New York. find
  18. Zébulon Goriely, Richard Diehl Martinez, Andrew Caines, Paula Buttery, and Lisa Beinborn. 2024. From babble to words: Pre-training language models on continuous streams of phonemes. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 37–53, Miami, FL,… find
  19. Noam Chomsky. 1980. Rules and Representations. Columbia University Press, New York. find
  20. Noam Chomsky and Howard Lasnik. 1977. Filters and control. Linguistic Inquiry, 8(3):425–504. find
  21. Noam Chomsky, Ian Roberts, and Jeffrey Watumull. 2023. The false promise of ChatGPT. The New York Times. Opinion. find
  22. John Hale. 2001. A probabilistic earley parser as a psycholinguistic model. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies, pages 1–8, Pittsburgh, Pennsylvania. Association for Computational Linguistics. tr…
  23. John Hale. 2016. Information-theoretical complexity metrics. Language and Linguistics Compass, 10(9):397–412. find
  24. Qian Niu, Junyu Liu, Ziqian Bi, Pohsun Feng, Benji Peng, Keyu Chen, Ming Li, Lawrence K. Q. Yan, Yichao Zhang, Caitlyn Heqi Yin, Cheng Fei, Tianyang Wang, Yunze Wang, Silin Chen, Ming Liu, Ziyuan Qin, Riyang Bao, Xinyuan Song, and Zekun Jiang. 2024. Large language models and cognitive science: A com… — [arXiv:2409.02387](https://arxiv.org/abs/2409.02387) find
  25. Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen, Chengxu Zhuang, Ryan Cotterell, Leshem Choshen, Alex Warstadt, and Ethan Gotlieb Wilcox. 2024. Findings of the second BabyLM challenge: Sample-efficient pretraining on developmentally plausible corpora. In The 2nd BabyLM Challen… find
  26. Kangsan Noh, Eunjeong Oh, and Sanghoun Song. 2024. Testing language models’ syntactic sensitivity to grammatical constraints: A case study of wanna contraction. Frontiers in Communication, 9:1442093. find
  27. Haerim Hwang. 2024. Wanna contraction in first language acquisition, child second language acquisition, and adult second language acquisition. Bilingualism: Language and Cognition, 27(3):322–333. find
  28. Steven T. Piantadosi. 2024. Modern language models refute Chomsky’s approach to language. In Edward Gibson and Moshe Poliak, editors, From Fieldwork to Linguistic Theory: A Tribute to Dan Everett, volume 15 of Empirically Oriented Theoretical Morphology and Syntax, chapter 15, pages 353–414. Languag… find
  29. Yasuko Ito. 2018. Acquisition of contraction constraint by Japanese learners of English. Journal of the Pan-Pacific Association of Applied Linguistics, 22(1):19– 41. find
  30. Paul M. Postal and Geoffrey K. Pullum. 1978. Traces and the description of English complementizer contraction. Linguistic Inquiry, 9(1):1–29. find
  31. Anna A. Ivanova, Aalok Sathe, Benjamin Lipkin, Un-nathi Kumar, Setayesh Radkani, Thomas H. Clark, Carina Kauf, Jennifer Hu, R. T. Pramod, Gabriel Grand, Vivian C. Paulun, Maria Ryskina, Ekin Akyürek, Ethan G. Wilcox, Nafisa Rashid, Leshem Choshen, Roger Levy, Evelina Fedorenko, Joshua B. Tenenbaum,… find
  32. Paul M. Postal and Geoffrey K. Pullum. 1982. The contraction debate. Linguistic Inquiry, 13(1):122– 138. find
  33. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  34. Roni Katzir. 2023. Why large language models are poor theories of human linguistic cognition: A reply to Piantadosi. Biolinguistics, 17:e13153. find
  35. Language, Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842–866.
  36. David Lightfoot. 1976. Trace theory and twice-moved NPs. Linguistic Inquiry, 7(4):559–582. find
  37. Joel T. Rotenberg. 1978. The Syntax of Phonology. Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, MA. find
  38. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535.
  39. David Samuel, Andrey Kutuzov, Lilja Øvrelid, and Erik Velldal. 2023. Trained on 100 million words and still in shape: BERT meets British National Corpus. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1954–1974, Dubrovnik, Croatia. Association for Computational Lingui… find
  40. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  41. Jean-Loup Tastet and Inar Timiryasov. 2024. BabyLlama-2: Ensemble-distilled models consistently outperform teachers with limited data. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 292–301, Miami, FL, USA. Association for Computational Linguisti… find
  42. Evan Lucas, Dylan Gaines, Tagore Rao Kosireddy, Kevin Li, and Timothy C. Havens. 2024. Using curriculum masking based on child language development to train a large language model with limited Nikitas Theodoropoulos, Giorgos Filandrianos, Vassilis Lyberatos, Maria Lymperaiou, and Giorgos Stamou. 202… find
  43. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377– 392.
  44. Ethan Gotlieb Wilcox, Richard Futrell, and Roger Levy. 2024. Using computational models to test syntactic learnability. Linguistic Inquiry, 55(4):805–848. find
  45. Xinru Yu, Bin Guo, Shiwei Luo, Jie Wang, Tao Ji, and Yuanbin Wu. 2024. AntLM: Bridging causal and masked language models. In The 2nd BabyLM Challenge at the 28th Conference on Computational Natural Language Learning, pages 324–331, Miami, FL, USA. Association for Computational Linguistics. find
  46. Inar Timiryasov and Jean-Loup Tastet. 2023. Baby Llama: Knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning, pages 279–289, Singapore. Asso… find
  47. Andrea Zukowski and Jaiva Larsen. 2011. Wanna contraction in children: Retesting and revising the developmental facts. Language Acquisition, 18(4):211– 241. find
  48. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30.
  49. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems, volume 32.
  50. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–3…
  51. Alex Warstadt and Samuel R. Bowman. 2022. What artificial neural networks can tell us about human language acquisition. In Shalom Lappin and Jean-Philippe Bernardy, editors, Algebraic Structures in Natural Language, pages 17–60. CRC Press, Boca Raton, FL.
  52. Alex Warstadt, Leshem Choshen, Aaron Mueller, Adina Williams, Ethan Wilcox, and Chengxu Zhuang. 2023a. Call for papers – the BabyLM challenge: Sample-efficient pretraining on a developmentally plausible corpus. arXiv preprint arXiv:2301.11796. find
  53. Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjape, Adina Williams, Tal Linzen, and Ryan Cotterell. 2023b. Findings of the BabyLM challenge: Sample-Efficient pretraining on developmentally plausible corpora. In Proceedings of th… find
  54. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/noh-2026-do-babylms-wanna-learn-wanna/