📰 BabyLM Workshop 2025: all accepted papers

41 papers · the most cited first · ← BabyLM Workshop 2025

1. Findings of the Third BabyLM Challenge: Accelerating Language Modeling Research with Cognitively Plausible Data ↗Lucas Charpentier, Leshem Choshen, Ryan Cotterell et al.2 citations
2. Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language ModelsRaha Askari, Sina Zarrieß, Özge Alacam et al.1 citation · our summary
3. Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning) ↗Francesca Padovani, Bastian Bunzeck, Manar Ali et al.1 citation
4. Influence-driven Curriculum Learning for Pre-training on Limited Data ↗Loris Schoenegger, Lukas Thoma, Terra Blevins et al.1 citation
5. Large Language Models and Children Have Different Learning Trajectories in Determiner Acquisition ↗Olivia La Fiandra, Nathalie Fernandez Echeverri, Patrick Shafto et al.1 citation
6. Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs ↗Lukas Edman, Alexander Fraser1 citation
7. Once Upon a Time: Interactive Learning for Storytelling with Small Language Models ↗Jonas Mayer Martins, Ali Hamza Bashir, Muhammad Rehan Khalid et al.1 citation
8. Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements ↗Patrick Haller, Jonas Golde, Alan Akbik1 citation
9. Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction ↗Suchir Salhan, Hongyi Gu, Donya Rooein et al.1 citation
10. A Comparison of Elementary Baselines for BabyLM ↗Rareș Păpușoi, Sergiu Nisioi
11. A Morpheme-Aware Child-Inspired Language Model ↗Necva Bölücü, Burcu Can
12. Active Curriculum Language Modeling over a Hybrid Pre-training Method ↗Eleni Fysikoudi, Sharid Loáiciga, Asad B. Sayeed
13. BLiSS: Evaluating Bilingual Learner Competence in Second Language Small Language Models ↗Yuan Gao, Suchir Salhan, Andrew Caines et al.
14. Babies Learn to Look Ahead: Multi-Token Prediction in Small LMsAnsar Aynetdinov, Alan Akbikour summary
15. Batch-wise Convergent Pre-training: Step-by-Step Learning Inspired by Child Language DevelopmentKo Yoshida, Daiki Shiono, Kai Sato et al.our summary
16. Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining ↗Matthew Theodore Roque, Dan John Velasco
17. BitMar: Low-Bit Multimodal Fusion with Episodic Memory for Edge DevicesEuhid Aman, Esteban Carlin, Hsing-Kuo Kenneth Pao et al.our summary
18. CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs ↗Luca Capone, Alessandro Bondielli, Alessandro Lenci
19. Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling ↗Jannek Ulm, Kevin Du, Vésteinn Snæbjarnarson
20. CurLL: A Developmental Framework to Evaluate Continual Learning in Language Models ↗Pavan Kalyan Tankala, Shubhra Mishra, Satya Lokam et al.
21. Design and Analysis of few Million Parameter Transformer-based Language Models trained over a few Million Tokens Dataset ↗Yen-Che Hsiao, Abhishek Dutta
22. Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models? ↗Arzu Burcu Güven, Anna Rogers, Rob van der Goot
23. Exploring smaller batch sizes for a high-performing BabyLM model architecture ↗Sharid Loáiciga, Eleni Fysikoudi, Asad B. Sayeed
24. FORGETTER with forgetful hyperparameters and recurring sleeps can continue to learn beyond normal overtfitting limits ↗Yamamoto Rui, Keiji Miura
25. Linguistic Units as Tokens: Intrinsic and Extrinsic Evaluation with BabyLM ↗Achille Fusco, Maria Letizia Piccini Bianchessi, Tommaso Sgrizzi et al.
26. Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling ↗Bianca-Mihaela Ganescu, Suchir Salhan, Andrew Caines et al.
27. Masked Diffusion Language Models with Frequency-Informed Training ↗Despoina Kosmopoulou, Efthymios Georgiou, Vaggelis Dorovatas et al.
28. MoEP: Modular Expert Paths for Sample-Efficient Language Modeling ↗Joonas Tapaninaho
29. Model Merging to Maintain Language-Only Performance in Developmentally Plausible Multimodal Models ↗Ece Takmaz, Lisa Bylinina, Jakub Dotlacil
30. Pretraining Language Models with LoRA and Artificial Languages ↗Nalin Kumar, Mateusz Lango, Ondrej Dusek
31. RecombiText: Compositional Data Augmentation for Enhancing LLM Pre-Training Datasets in Low-Resource Scenarios ↗Alexander Tampier, Lukas Thoma, Loris Schoenegger et al.
32. Rethinking the Role of Text Complexity in Language Model Pretraining ↗Dan John Velasco, Matthew Theodore Roque
33. Single layer tiny Co4 outpaces GPT-2 and GPT-BERT ↗Noor Ul Zain, Mohsin Raza Naseem, Ahsan Adeel
34. SlovakBabyLM: Replication of the BabyLM and Sample-efficient Pretraining for a Low-Resource Language ↗Ľuboš Kriš, Marek Suppa
35. TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in HebrewAnita Gelboim, Elior Sulemour summary
36. Two ways into the hall of mirrors: Language exposure and lossy memory drive cross-linguistic grammaticality illusions in language models ↗Kate McCurdy, Katharina Christian, Amelie Seyfried et al.
37. Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modeling ↗Hyunji Lee, Wenhao Yu, Hongming Zhang et al.
38. Unifying Mixture of Experts and Multi-Head Latent Attention for Efficient Language Models ↗Sushant Mehta, Raj Dandekar, Rajat Dandekar et al.
39. What did you say? Generating Child-Directed Speech Questions to Train LLMsWhitney Poh, Michael Tombolini, Libby Barakour summary
40. What is the Best Sequence Length for BabyLM? ↗Suchir Salhan, Richard Diehl Martinez, Zébulon Goriely et al.
41. You are an LLM teaching a smaller model everything you know: Multi-task pretraining of language models with LLM-designed study plans ↗Wiktor Kamzela, Mateusz Lango, Ondrej Dusek

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/babylm/2025/papers/