📄 Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition

Summary

LLMs need far more training data than children to reach comparable linguistic competence. Drawing on the Critical Period Hypothesis and Newport's Less-is-More Hypothesis, the authors test whether mimicking how children's working memory grows can make LM training more data-efficient. They model working-memory growth across development as an exponential curve of the form y = b − aˣ (0 < a < 1), which rises steeply early and then levels off at an adult ceiling. They turn this curve into a training schedule for GPT-2 called DynamicLimit-Exp. Memory is tightly restricted at the start of training and the restriction is loosened exponentially over time. The model is trained on the child-directed speech corpus AO-CHILDES and also on Wikipedia. It is evaluated with Zorro, a targeted syntactic benchmark designed for child-directed speech. DynamicLimit-Exp beats both an unconstrained baseline and a fixed-limit baseline. The result points to a cognitively motivated inductive bias for data-efficient LMs. It also gives indirect, computational-level support for working-memory development as a mechanism behind the critical period.

Key points

• The schedule is grounded in three developmental stages of human working memory: 2–7 years (rapid growth), 8–14 years (slowing growth) and 15+ years (adult plateau). The authors argue that linear or logarithmic curves cannot capture both the fast early rise and the eventual ceiling.
• The baselines are NoLimit (no memory constraint) and a static memory limit. DynamicLimit-Exp overtakes both by epoch 10 in several Zorro categories, with the clearest late-stage gains on CASE and FILLER.GAP.
• In embedding analyses using t-SNE, entropy and inter-cluster distance, NoLimit's representations collapse after epoch 5. DynamicLimit-Exp keeps embeddings more dispersed and continues to separate them across epochs.
• A sentence-length test used four Wikipedia sets of 500,000 sentences each. DynamicLimit-Exp was significantly better for lengths 11–50 (58.7 vs 47.0), 51–100 (42.5 vs 40.6) and 101–150 (40.8 vs 37.3). NoLimit was marginally ahead only on 5–10-word sentences (47.2 vs 46.8).
• The code is released as CPLM at github.com/osekilab/CPLM. The paper appeared at ACL 2025 (Long Papers).