Elamur Research⌕
☰
home / papers / 2026 / ChildEval:WHEN LARGE LANGUAGE MODELS MEET CHILDREN'S PERSONALITIES

ChildEval:WHEN LARGE LANGUAGE MODELS MEET CHILDREN'S PERSONALITIES

Paper · 2026 · Yanyan Luo, Xue Han, Chunxu Zhao et al.

Summary

LLM-based chatbots are increasingly used as companions for young children. Yet nobody has systematically tested whether these models pick up on and respect a child's preferences. Existing personalization benchmarks mostly target adults, and child-focused ones cover only narrow aspects such as safety or style imitation. The authors present ChildEval, a synthetic benchmark of 29K preschooler personas (ages 3–6) generated with Qwen2.5-72B and deduplicated with FAISS. Each persona is paired with preferences that may agree with, contradict, or be unrelated to the persona. Every preference appears in two forms: a single direct statement (explicit) and a 6–10-turn dialogue from which it must be inferred (implicit). The preference sits inside long conversation histories padded with unrelated sessions. Evaluation combines Preference Consistency with new child-oriented criteria. Open-source LLMs lose track of preferences over long contexts, particularly implicit ones. Finetuning on ChildEval improves adherence but introduces new kinds of inconsistency. The resource is a controlled diagnostic tool for child-centered personalization.

Key points

  • Data scale and filtering: 29K personas and 58K generated explicit preferences, of which 46K were kept after FAISS-based semantic filtering. Names and identifiers were removed through LLM processing and human review.
  • Preference taxonomy: 5 top-level and 14 sub-level topics, based on the 2012 guidelines of China's Ministry of Education. "Cognitive development and exploration" is the largest topic at 29.20%.
  • Metrics: Preference Consistency (PC) plus Child-Oriented Evaluation (COE), which has four sub-dimensions:
    • Emotional Adaptation (EA)
    • Interaction Scaffolding (IS)
    • Developmental Appropriateness (DA)
    • Engagement (EG)
  • Error analysis: Preference-Unaware Violations dominate at first and grow with longer irrelevant context. LoRA and PSM finetuning reduce them but shift errors elsewhere. LoRA produces more Inconsistency Violations, while PSM yields 79% Unhelpful Responses on implicit data. Finetuning gives the largest gains in IS.
  • Availability and limits: Code and data are released at github.com/ziyanluo/ChildEval. The benchmark is fully synthetic and does not model child-specific safety risks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢China MobileBeijing · China9 works · 21 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yanyan Luo, Xue Han, Chunxu Zhao, Ruiqiao Bai, Yaxing Zhang, Qian Hu, Lijun Mei and Junlan Feng. 2026. ChildEval:WHEN LARGE LANGUAGE MODELS MEET CHILDREN'S PERSONALITIES. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1070

@inproceedings{luo2026childeval,
  title = {{ChildEval:WHEN LARGE LANGUAGE MODELS MEET CHILDREN'S PERSONALITIES}},
  author = {Yanyan Luo, Xue Han, Chunxu Zhao, Ruiqiao Bai, Yaxing Zhang, Qian Hu, Lijun Mei and Junlan Feng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1070},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1070}
}

References

49 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. find
  2. Alissa N Antle. 2008. Child-based personas: need, ability and experience. Cognition, Technology & Work, 10(2):155–166. find
  3. Ruiqiao Bai, Xue Han, Shuo Lei, Junlan Feng, Yanyan Luo, and Chao Deng. 2025. Self-attention-based graph-of-thought for math problem solving. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6112–6125, Vienna, Austria. Association for Computational Linguistics. find
  4. Marco Braga, Pranav Kasela, Alessandro Raganato, and Gabriella Pasi. 2024. Synthetic data generation with large language models for personalized community question answering. In 2024 IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), pages 360–366. IEEE. find
  5. Jiaju Chen, Minglong Tang, Yuxuan Lu, Bingsheng Yao, Elissa Fan, Xiaojuan Ma, Ying Xu, Dakuo Wang, Yuling Sun, and Liang He. 2025a. Characterizing llm-empowered personalized story reading and interaction for children: Insights from multi-stakeholder perspectives. In Proceedings of the 2025 CHI Confe… find
  6. Jiaju Chen, Minglong Tang, Yuxuan Lu, Bingsheng Yao, Elissa Fan, Xiaojuan Ma, Ying Xu, Dakuo Wang, Yuling Sun, and Liang He. 2025b. Characterizing llm-empowered personalized story reading and interaction for children: Insights from multi-stakeholder perspectives. Conference on Human Factors in Compu… find
  7. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library.
  8. Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani. 2024. Longlamp: A benchmark for personalized long-form text generati… — [arXiv:2407.11016](https://arxiv.org/abs/2407.11016) find
  9. Arafat Md Easin, Saha Sourav, and Orosz Tamás. 2024. An intelligent llm-powered personalized assistant for digital banking using langgraph and chain of thoughts. In 2024 IEEE 22nd Jubilee International Symposium on Intelligent Systems and Informatics (SISY), pages 625–630. IEEE. find
  10. Juntao Li, Chang Liu, Chongyang Tao, Zhangming Chan, Dongyan Zhao, Min Zhang, and Rui Yan. 2021. Dialogue history matters! personalized response selection in multi-turn retrieval-based chatbots. ACM Transactions on Information Systems (TOIS), 39(4):1– 25. find
  11. Tiantian Feng, Anfeng Xu, Rimita Lahiri, Helen Tager-Flusberg, So Hyun Kim, Somer Bishop, Catherine Lord, and Shrikanth Narayanan. 2024. Can generic llms help analyze child-adult interactions involving children with autism in clinical observation? Preprint, arXiv:2411.10761. — [arXiv:2411.10761](https://arxiv.org/abs/2411.10761) find
  12. Xinyu Li, Zachary C. Lipton, and Liu Leqi. 2024. Personalized language modeling from personalized human feedback. CoRR, abs/2402.05133. — [arXiv:2402.05133](https://arxiv.org/abs/2402.05133) find
  13. Gemini, :, Rohan Anil, Sebastian Borgeaud, and et al. 2025. Gemini: A family of highly capable multimodal models. Preprint, arXiv:2312.11805.
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  15. Xue Han, Yi-Tong Wang, Jun-Lan Feng, Chao Deng, Zhan-Heng Chen, Yu-An Huang, Hui Su, Lun Hu, and Peng-Wei Hu. 2023. A survey of transformer-based multimodal pre-trained modals. Neurocomputing, 515:89–106. find
  16. Xue Han, Yitong Wang, Junlan Feng, wenchun.gao, Qian Hu, and Chao Deng. 2025. Loire: Lifelong learning on incremental data via pre-trained language model growth efficiently. In International Conference on Learning Representations, volume 2025, pages 1024–1044. find
  17. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
  18. Bernd Huber, Ghazal Fazelnia, Andreas Damianou, Sebastian Peleato, Max Lefarov, Praveen Ravichandran, Marco De Nadai, Mounia Lalmas-Roellke, and Paul N Bennett. 2025. Embedding-to-prefix: Parameter-efficient personalization for pre-trained large language models. arXiv preprint arXiv:2505.17051. — [arXiv:2505.17051](https://arxiv.org/abs/2505.17051) find
  19. Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle H. Ungar, Camillo J. Taylor, and Dan Roth. 2025. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale. CoRR, abs/2504.14225. — [arXiv:2504.14225](https://arxiv.org/abs/2504.14225) find
  20. Jiahong Liu, Zexuan Qiu, Zhongyang Li, Quanyu Dai, Jieming Zhu, Minda Hu, Menglin Yang, and Irwin King. 2025. A survey of personalized large language models: Progress and future directions. arXiv preprint arXiv:2502.11528. — [arXiv:2502.11528](https://arxiv.org/abs/2502.11528) find
  21. Jing Liu and Abdellah Fourtassi. 2024. Benchmarking llms for mimicking child-caregiver language in interaction. arXiv preprint arXiv:2412.09318. find
  22. Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. 2022. Memory-assisted prompt editing to improve GPT-3 after deployment. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2833–286… find
  23. Lucie Charlotte Magister, Katherine Metcalf, Yizhe Zhang, and Maartje ter Hoeve. 2024. On the way to llm personalization: Learning to remember user conversations. arXiv preprint arXiv:2411.13405. find
  24. MOE. 2012. Notice of the ministry of education on issuing the guidelines for learning and development of children aged 3–6. Ministry of Education of the People’s Republic of China, Document No. find
  25. Jisoo Mok, Ik hwan Kim, Sangkwon Park, and Sungroh Yoon. 2025. Exploring the potential of llms as personalized assistants: Dataset, evaluation, and analysis. Preprint, arXiv:2506.01262. find
  26. Mir Tafseer Nayeem and Davood Rafiei. 2024. KidLM: Advancing language models for children – early insights and future directions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4813–4836, Miami, Florida, USA. Association for Computational Linguistics… find
  27. OpenAI, :, Aaron Jaech, Adam Kalai, and et al. 2024. Openai o1 system card. Preprint, arXiv:2412.16720.
  28. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Jianfeng Gao. 2025. On memory construction and retrieval for personalized conversational agents. Preprint, arXiv:2502.05589. 2025. MultiPL-MoE: Multi-programming-ling… find
  29. Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Defu Lian, Zhicheng Dou, and Tiejun Huang. 2025. Memorag: Boosting long context processing with global memory-enhanced retrieval augmentation. In Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025-2 May 2… find
  30. Bin Wu, Zhengyan Shi, Hossein A Rahmani, Varsha Ramineni, and Emine Yilmaz. 2024. Understanding the role of user profile in the personalization of large language models. arXiv preprint arXiv:2406.17803. — [arXiv:2406.17803](https://arxiv.org/abs/2406.17803) find
  31. Yilun Qiu, Xiaoyan Zhao, Yang Zhang, Yimeng Bai, Wenjie Wang, Hong Cheng, Fuli Feng, and Tat-Seng Chua. 2025. Measuring what makes you unique: Difference-aware user modeling for enhancing llm personalization. arXiv preprint arXiv:2503.02450. find
  32. Prasanjit Rath, Hari Shrawgi, Parag Agrawal, and Sandipan Dandapat. 2025. Llm safety for children. Preprint, arXiv:2502.12552. find
  33. Michael J. Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Held, and Diyi Yang. 2025. SynthesizeMe! inducing persona-guided prompts for personalized reward models in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p… find
  34. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2023. Lamp: When large language models meet personalization. arXiv preprint arXiv:2304.11406.
  35. Woosuk Seo, Chanmo Yang, and Young-Ho Kim. 2024. Chacha: Leveraging large language models to prompt children to share their emotions about personal events. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI ’24, page 1–20. ACM. find
  36. Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024. Democratizing large language models via personalized parameter-efficient fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 1…
  37. Maria Valentini, Jennifer Weber, Jesus Salcido, Téa Wright, Eliana Colunga, and Katharina Kann. 2023. On the automatic generation and simplification of children’s stories. arXiv preprint arXiv:2310.18502. find
  38. Ping Wang. 2013. Interpretation of the Learning and Development Guidelines for Children Aged 3-6. Northeast Normal University Press. find
  39. Qing Wang, Xue Han, Jiahui Wang, Lehao Xing, Qian Hu, Lianlian Zhang, Chao Deng, and Junlan Feng. find
  40. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597.
  41. Xuhai Xu, Bingsheng Yao, Ziqi Yang, Shao Zhang, Ethan Rogers, Stephen Intille, Nawar Shara, Guodong Gao, and Dakuo Wang. 2024. Talk2care: Facilitating asynchronous patient-provider communication with large-language-model. In Proceedings of the AAAI Symposium Series, pages 146–151. find
  42. Yiyan Xu, Jinghao Zhang, Alireza Salemi, Xinting Hu, Wenjie Wang, Fuli Feng, Hamed Zamani, Xiangnan He, and Tat-Seng Chua. 2025. Personalized generation in large model era: A survey. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p…
  43. Jiarui Zhang. 2024. Guided profile generation improves personalization with llms. arXiv preprint arXiv:2409.13093. — [arXiv:2409.13093](https://arxiv.org/abs/2409.13093) find
  44. Xinliang Frederick Zhang, Nick Beauchamp, and Lu Wang. 2025a. PRIME: large language model personalization with cognitive memory and thought processes. CoRR, abs/2507.04607. — [arXiv:2507.04607](https://arxiv.org/abs/2507.04607) find
  45. Yijing Zhang, Dyah Adila, Changho Shin, and Frederic Sala. 2025b. Personalize your LLM: fake it then align it. In Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, pages 7287– 7301. Association for Computational Linguistics. find
  46. Siyan Zhao, Mingyi Hong, Yang Liu, Devamanyu Hazarika, and Kaixiang Lin. 2025a. Do llms recognize your preferences? evaluating personalized preference following in llms. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  47. Weixiang Zhao, Xingyu Sui, Yulin Hu, Jiahe Guo, Haixiao Liu, Biye Li, Yanyan Zhao, Bing Qin, and Ting Liu. 2025b. Teaching language models to evolve with users: Dynamic profile modeling for personalized alignment. CoRR, abs/2505.15456. — [arXiv:2505.15456](https://arxiv.org/abs/2505.15456) find
  48. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. Preprint, arXiv:2405.01470.
  49. Yuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang, Qifan Wang, Chao Zhang, and Bo Dai. 2024. HYDRA: model factorization framework for black-box LLM personalization. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, V… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/luo-2026-childeval-when-large-language-models/