Elamur Research⌕
☰
home / papers / 2026 / Why Did Apple Fall: Evaluating Curiosity in Large Language Models

Why Did Apple Fall: Evaluating Curiosity in Large Language Models

Paper · 2026 · Haoyu Wang, Sihang Jiang, Yuyan Chen et al.

Summary

Curiosity-inspired intrinsic rewards are increasingly used in RL training of LLMs, but it has not been checked whether these models actually behave in curious ways, or whether the reported gains simply come from amplified statistical signals such as entropy. The authors build a psychology-based evaluation framework. They adapt the Five-Dimensional Curiosity Scale Revised (5DCR) for LLM self-reports. They then test the questionnaire results with scenario-based behavioral tasks adapted from human psychology experiments. These cover three dimensions: information seeking, thrill seeking and social curiosity.

The models show human-like patterns that are consistent between questionnaires and behavior. However, the profile is lopsided: information seeking is strong while thrill seeking is weak. It also shifts with prompts and personas. The authors read this as a byproduct of fitting human data and safety tuning, not a genuine internal drive.

Building on this, they propose Curious Chain-of-Questioning (CoQ), which has models pose and answer auxiliary questions while reasoning. CoQ outperforms standard CoT on DetectBench and Math. The work suggests that imitating curious behavior can improve reasoning even without a real curiosity trait.

Key points

  • The 5DCR adaptation covers 24 items on a 7-point Likert scale and six subdimensions: JE, DS, ST, TS, OSC and CSC. Comparison with human norms uses Cohen's d, and reliability is measured with McDonald's omega.
  • The behavioral probes include:
    • a missing-letter word game, where models choose whether to reveal the answer, as a measure of information seeking;
    • an undersea window-choice scenario for thrill seeking;
    • a free conversation with a stranger for social curiosity.
  • A logit-lens analysis of persona prompts on Qwen2.5-7B finds that the "curiosity" score rises clearly only in the final four layers.
  • With prompting only, CoQ lifts GPT-4o-mini from 38.2 to 45.2 on DetectBench (+7.0, versus +4.6 for CoT) and improves Math by +6.3 (versus +4.2 for CoT).
  • Fine-tuning open-source models on Curious CoQ traces gives the best results: Qwen2.5-7B gains +23.7 on DetectBench and +12.4 on Math, and Llama3-8B gains +33.5 on DetectBench. These runs use more thinking tokens. Code is available in the GitHub repository CuriosityEval.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cornell UniversityIthaca · United States120 works · 59,940 citations🎓Fudan UniversityShanghai · China167 works · 20,987 citations🏢Huawei TechnologiesShenzhen · China75 works · 12,975 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Haoyu Wang, Sihang Jiang, Yuyan Chen, Yitong Wang, Xiaojun Meng, Jiansheng Wei and Yanghua Xiao. 2026. Why Did Apple Fall: Evaluating Curiosity in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1954

@inproceedings{wang2026why,
  title = {{Why Did Apple Fall: Evaluating Curiosity in Large Language Models}},
  author = {Haoyu Wang, Sihang Jiang, Yuyan Chen, Yitong Wang, Xiaojun Meng, Jiansheng Wei and Yanghua Xiao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1954},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1954}
}

References

45 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025. The unreasonable effectiveness of entropy minimization in llm reasoning. arXiv preprint arXiv:2505.15134. find
  2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  3. Ali Günay Balım. 2009. The effects of discovery learning on students’ success and inquiry learning skills. Eurasian Journal of Educational Research (EJER), (35). find
  4. Daniel E Berlyne. 1966. Curiosity and exploration: Animals spend much of their time seeking stimuli whose significance raises problems for psychology. Science, 153(3731):25–33. find
  5. Daniel Ellis Berlyne and 1 others. 1954. A theory of human curiosity. find
  6. Ravinder Bonniga and Dr AB Saraswathi. 2020. Literature review of cronbachalphacoefficient and mcdonald’s omega coefficient. European Journal of Molecular & Clinical Medicine, 7(06). find
  7. Gregory J Boyle. 1995. Myers-briggs type indicator (mbti): some psychometric limitations. Australian Psychologist, 30(1):71–74. find
  8. Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, and 1 others. 2024. From persona to personalization: A survey on role-playing language agents. arXiv preprint arXiv:2404.18231.
  9. Robert P Collins, Jordan A Litman, and Charles D Spielberger. 2004. The measurement of perceptual curiosity. Personality and individual differences, 36(5):1127–1141. find
  10. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  11. Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, and 1 others. 2025. Stepwise perplexity-guided refinement for efficient chain-of-thought reasoning in large language models. arXiv preprint arXiv:2502.13260. find
  12. Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, and 1 others. 2025. Cde: Curiosity-driven exploration for efficient reinforcement learning in large language models. arXiv preprint arXiv:2509.09675. — [arXiv:2509.09675](https://arxiv.org/abs/2509.09675) find
  13. Marc J Diener. 2010. Cohen’s d. The Corsini encyclopedia of psychology, pages 1–1. find
  14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  15. Jingtong Gao, Ling Pan, Yejing Wang, Rui Zhong, Chi Lu, Qingpeng Cai, Peng Jiang, and Xiangyu Zhao. 2025. Navigate the unknown: Enhancing llm reasoning with intrinsic motivation guided exploration. arXiv preprint arXiv:2505.17621. — [arXiv:2505.17621](https://arxiv.org/abs/2505.17621) find
  16. Lingyu Gao, Debanjan Ghosh, and Kevin Gimpel. 2022. “what makes a question inquisitive?” a study on type-controlled inquisitive question generation. In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 240–257. find
  17. Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, and 1 others. 2024. Detectbench: Can large language model detect and piece together implicit evidence? arXiv preprint arXiv:2406.12641. find
  18. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  19. Pengrui Han, Rafal Kocielnik, Peiyang Song, Ramit Debnath, Dean Mobbs, Anima Anandkumar, and R Michael Alvarez. 2025. The personality illusion: Revealing dissociation between self-reports & behavior in llms. arXiv preprint arXiv:2509.03730. — [arXiv:2509.03730](https://arxiv.org/abs/2509.03730) find
  20. Donna Harrington. 2009. Confirmatory factor analysis. Oxford university press. find
  21. Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, and 1 others. 2020. Array programming with numpy. Nature, 585(7825):357–362.
  22. Freda-Marie Hartung and Britta Renner. 2011. Social curiosity and interpersonal perception: A judge× trait interaction. Personality and Social Psychology Bulletin, 37(6):796–814. find
  23. John D Hunter. 2007. Matplotlib: A 2d graphics environment. Computing in science & engineering, 9(03):90–95. find
  24. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  25. Shashidhar Reddy Javaji and Zining Zhu. 2024. What would you ask when you first saw a2+b2=c2? evaluating llm on curiosity-driven questioning. In 2nd AI4Research Workshop: Towards a Knowledge-grounded Scientific Research Lifecycle. find
  26. Jamie Jirout and David Klahr. 2012. Children’s scientific curiosity: In search of an operational definition of an elusive concept. Developmental review, 32(2):125–160. find
  27. Todd B Kashdan, David J Disabato, Fallon R Goodman, and Patrick E McKnight. 2020. The five-dimensional curiosity scale revised (5dcr): Briefer subscales while separating overt and covert social curiosity. Personality and individual differences, 157:109836. find
  28. Todd B Kashdan, Matthew W Gallagher, Paul J Silvia, Beate P Winterstein, William E Breen, Daniel Terhar, and Michael F Steger. 2009. The curiosity and exploration inventory-ii: Development, factor structure, and psychometrics. Journal of research in personality, 43(6):987–998. find
  29. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):… find
  30. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  31. George Loewenstein. 1994. The psychology of curiosity: A review and reinterpretation. Psychological bulletin, 116(1):75. find
  32. Georgy Meshcheryakov, Anna A Igolkina, and Maria G Samsonova. 2021. semopy 2: A structural equation modeling package with random effects in python. arXiv preprint arXiv:2106.01140. — [arXiv:2106.01140](https://arxiv.org/abs/2106.01140) find
  33. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, and 1 others. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  34. Thomas Scialom and Jacopo Staiano. 2020. Ask to learn: A study on curiosity-driven question generation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2224– 2235. find
  35. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  36. Aditya Singh and Jaison A Manjaly. 2021. The effect of information gap and uncertainty on curiosity and its resolution. Journal of Cognitive Psychology, 33(4):403–423. find
  37. LA Snider and SE Swedo. 2004. Pandas: current status and directions for research. Molecular psychiatry, 9(10):900–907. find
  38. Haoran Sun, Yekun Chai, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. 2025. Curiosity-driven reinforcement learning from human feedback. arXiv preprint arXiv:2501.11463. find
  39. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  40. InternLM Team. 2023. Internlm: A multilingual language model with progressively enhanced capabilities. find
  41. Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, and 1 others. 2020. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261–272.
  42. Michael L Waskom. 2021. Seaborn: statistical data visualization. Journal of Open Source Software, 6(60):3021. on the MS-Swift (Zhao et al., 2025) framework. For inference, the default parameters of the framework were adopted. The parameters used during SFT and GRPO are shown in table 5 and 6. The re… find
  43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  44. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and 1 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural lan…
  45. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-why-did-apple-fall-evaluating/