Elamur Research⌕
☰
home / papers / 2026 / ChineseDevBench: A Chinese Developmental Benchmark for Language Development

ChineseDevBench: A Chinese Developmental Benchmark for Language Development

Summary

Data-efficient language models (e.g., BabyLM) are trained on roughly child-sized input, yet they are mostly evaluated with adult-oriented English benchmarks and seldom compared with real child behavior. The authors address this gap for Mandarin with ChineseDevBench, a suite of eight tasks. Five draw on child behavioral data (analogy, word association, verbal fluency, age of acquisition, spontaneous production), and three use adult norms (sentence similarity, word association, perceptual feature ratings). They train Chinese GPT-2 models on child-scale Mandarin material, including cartoon transcripts, picture books, audiobooks and child-directed speech, presented in an age-ordered curriculum. They then measure model–human agreement at successive checkpoints, mainly through Spearman correlation and RSA. Agreement generally rises as training proceeds. However, the age band of a curriculum stage does not explain how well the model performs, and the model's syntax does not become more complex the way children's does. The benchmark gives researchers a non-English tool for locating where distributional learning matches child development and where it departs from it.

Key points

  • Benchmark composition: five child-oriented tasks (three built on unreleased data, two on public data) and three public adult tasks. The child data include CHILD-ANALOGY (117 children aged 3–6, 15 items) and CHILD-ASSOC (733 children aged 2.5–7.5). The adult tasks use ChineseSTS (14,743 sentence pairs) and SWOW-ZH (10,192 cues, more than 2 million responses).
  • Age-of-acquisition and production data: model AoA is estimated from surprisal learning curves, following Chang and Bergen (2022), and compared with Wordbank (309 words kept) and Liu et al. (2011) norms (66 words kept). Production is compared against CHILDES Mandarin and CPCSLD (648 kindergarten children).
  • Training corpus: about 2,000 Bilibili/YouTube cartoon videos transcribed with Whisper, 1,158 digitized storybooks for ages 0–6, and recordings of child-directed speech.
  • Lexical findings: the model picks up function words quickly but is slow on embodied vocabulary such as action verbs and sensory adjectives. Unlike reported English results, it learns nouns early and verbs later.
  • Adult tasks and availability: association tasks show low correlations, and the Time and Emotion dimensions are hard to align. Data and code are released at github.com/Evaisgreat/Chinese-DevBench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2an6vr7tp5hn

@inproceedings{wang2026chinesedevbench,
  title = {{ChineseDevBench: A Chinese Developmental Benchmark for Language Development}},
  author = {Wang, Shaonan and Wu, Yiwen and Li, Na and Chen, Zesheng and Wang, Gan and Zhang, Shuchen and Sun, Xin and Li, Luan and Chen, Yaran},
  booktitle = {Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics},
  year = {2026},
  doi = {10.63317/2an6vr7tp5hn},
  url = {https://doi.org/10.63317/2an6vr7tp5hn}
}

References

20 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Early grammatical marking development in mandarin-speaking toddlers. Melissa Bowerman and Stephen C Levinson. Developmental psychology, 58(4):631. 2001. Language acquisition and conceptual de-Paul Ibbotson. 2013. The scope of usage-based velopment. 3. Cambridge University Press. theory. Frontiers in… find
  2. Tyler A Chang and Benjamin K Bergen. 2022. find
  3. Bing Li, Ziyi Ding, Simon De Deyne, and Qing Word acquisition in neural language models. find
  4. Cai. 2024. A large-scale database of mandarin Transactions of the Association for Computa-chinese word associations from the small world tional Linguistics, 10:1–16. of words project. Behavior Research Methods, Lucas Georges Gabriel Charpentier, Leshem 57(1):34. find
  5. Choshen, Ryan Cotterell, Mustafa Omer Gul, Luan Li and Thomas T. Hills. 2026. Exploration-Michael Y Hu, Jing Liu, Jaap Jumelet, Tal versus-exploitation of semantic networks pro-Linzen, Aaron Mueller, Candance Ross, et al. motes early lexical development: Evidence from 2025. Findings of the third bab… find
  6. First BabyLM Workshop, pages 399–420. Youyi Liu, Meiling Hao, Ping Li, and Hua Shu. find
  7. Noam Chomsky. 2002. Syntactic structures. Wal-2011. Timed picture naming norms for mandarin ter de Gruyter. chinese. PloS one, 6(1):e16505. find
  8. Chen Feng, Shuo Wang, and Shuang Li. 2026. find
  9. Weiyi Ma, Roberta Michnick Golinkoff, Kathy Cpcsld: A lexical database of chinese preschool Hirsh-Pasek, Colleen McDonough, and Twila children’s spoken words. Behavior Research Tardif. 2009. Imageability predicts the age of ac-Methods, 58(2):54. quisition of verbs in chinese children. Journal of Rob… find
  10. Min Hao, Hua Shu, Aiqing Xing, and Ping Li. 2008. find
  11. Julius Steuer, Marius Mosbach, and Dietrich Early vocabulary inventory for mandarin chi-Klakow. 2023. Large gpt-like models are bad ba-nese. Behavior Research Methods, 40(3):728– bies: A closer look at the relationship between 733. linguistic competence and psycholinguistic mea-Michael Y Hu, Aaron M… find
  12. Cotterell, Leshem Choshen, Alex Warstadt, and Ethan Gotlieb Wilcox. 2024. find
  13. Xin Sun, Kehui Zhang, Rebecca A Marks, Nia Nick-Findings of the second babylm challenge: Sample-efficient pretraining mentally plausible corpora. on erson, Rachel L Eggleston, Chi-Lin Yu, Tai-Li develop-Chou, Twila Tardif, and Ioulia Kovelman. 2022. arXiv preprint What’ s in a word? cross-linguistic… find
  14. Alvin Tan, Chunhua Yu, Bria Long, Wanjing Ma, Tonya Murray, Rebecca Silverman, Jason Yeatman, and Michael C Frank. 2024. Devbench: A multimodal developmental benchmark for language learning. Advances in Neural Information Processing Systems, 37:77445–77467. find
  15. Tempo Po-Yi Tang, Dustin Kai-Yan Lau, and Man-Tak Leung. 2023. find
  16. Corpus of mandarin child language: a preliminary study on the acquisition of semantic content categories in mandarin-speaking preschoolers. Frontiers in psychology, 14:1234525. find
  17. Michael Tomasello. 2003. find
  18. Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press. Shaonan Wang, Yunhao Zhang, Weiting Shi, Guangyao Zhang, Jiajun Zhang, Nan Lin, and Chengqing Zong. 2023. A large dataset of semantic ratings and its computational extension. find
  19. Scientific Data, 10(1):106. Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, et al. 2023. Findings of the babylm challenge: Sample-efficient pretraining on developmentally plausible corpora. In Pro… find
  20. 车 (car) 火车 (train) 阿姨 (auntie) 今天 (today) 告诉 (tell) 奶奶 (grandma) 喜欢 (like) 小朋友 (kids) 桌子 (table) 现在 (now) 朋友 (friend) 张 (classifier) 干嘛 (what for) 长 (grow/long) 漂亮 (pretty) 就 (then) 已经 (already) 叔叔 (uncle) 飞机 (airplane) 嘛 (particle) Model AoA Word Child AoA Model Earliest 2*Rank final_particles vehi… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-chinesedevbench-chinese-developmental-benchmark-language/