Elamur Research⌕
☰
home / papers / 2026 / KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions

KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions

Paper · 2026 · Tingyu Wu, Zhisheng Chen, Ziyan Weng et al.

Summary

Digital companions meant to support a user for years must do more than recall facts. They need to explain and anticipate behaviour from a person's motivations and values. Current long-term memory benchmarks mostly use chat logs or synthetic histories and mainly test retrieval. KnowMe-Bench instead draws on long autobiographical narratives (~4.7M tokens), which interleave actions, context and inner thoughts. A multi-agent pipeline with verify-and-revise checks and human expert validation turns each narrative into a time- and place-anchored cognitive stream. Flashbacks are moved to their original point in the chronology ("mnestic realignment"). Questions are evidence-linked and range from factual recall to attributing mental states and reasoning about decision principles. On several LLM backbones, retrieval-style memory (Naive RAG, Mem0) mostly boosts fact-level scores. Chronology and insight tasks remain weak, with MemOS helping more consistently. The benchmark shows that person understanding needs memory with explicit temporal provenance, not just a searchable fact store.

Key points

  • Each narrative is split into five textual fields: visual observations, auditory inputs, situational context, background knowledge, and inner monologue. The pipeline runs in four stages: segmentation, atomic unit (ANU) extraction, timeline generation, and narrative generation.
  • Evaluation has seven tasks in three levels:
    • T1 Extract and T2 Abstain make up Level I, which focuses on Fact & Entity.
    • Level II, Narrative Logic: T3 Temporal, T4 Order, T5 Trigger.
    • Level III, Insight: T6 Mind–Body, T7 Principle.
  • Tested backbones were Qwen3-32B, GPT-5-mini, DeepSeek-R1 and Gemini-3 Pro, each with Base, Naive RAG, Mem0 and MemOS variants. A-Mem was tested on Qwen3-32B only.
  • An "update paradox" appears: state-updating memory can treat recalled past events as overwrites of the present state. For example, Qwen3-32B + A-Mem scores 55.7 on T6 but drops to 16.0 on T3.
  • In closed-book contamination checks, Level II accuracy falls from 89.1 to 33.2 after de-identification. Level III stays at 0, so memorized literary knowledge does not explain the results.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore180 works · 34,304 citations🎓The University of Texas Health Science Center at HoustonHouston · United States5 works · 36 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 works

Cite

Cite as Tingyu Wu, Zhisheng Chen, Ziyan Weng, Shuhe Wang, Shuo Zhang, Sen Hu, Silin Wu, Qizhen Lan, Huacan Wang and Ronghao Chen. 2026. KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1394

@inproceedings{wu2026knowme,
  title = {{KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions}},
  author = {Tingyu Wu, Zhisheng Chen, Ziyan Weng, Shuhe Wang, Shuo Zhang, Sen Hu, Silin Wu, Qizhen Lan, Huacan Wang and Ronghao Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1394},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1394}
}

References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jocelyn Brickman, Mehak Gupta, and Joshua R Olt-manns. 2025. Large language models for psychological assessment: A comprehensive overview. Advances in Methods and Practices in Psychological Science, 8(3):25152459251343582. find
  2. David Castillo-Bolado, Joseph Davidson, Finlay Gray, and Marek Rosa. 2024. Beyond prompts: Dynamic conversational benchmarking of large language models. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. find
  3. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509. — [arXiv:2507.21509](https://arxiv.org/abs/2507.21509) find
  4. Xize Cheng, Dongjie Fu, Xiaoda Yang, Minghui Fang, Ruofan Hu, Jingyu Lu, Bai Jionghao, Zehan Wang, Shengpeng Ji, Rongjie Huang, and 1 others. 2025. Omnichat: Enhancing spoken dialogue systems with scalable synthetic data for diverse scenarios. arXiv preprint arXiv:2501.01384. — [arXiv:2501.01384](https://arxiv.org/abs/2501.01384) find
  5. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. Preprint, arXiv:2504.19413.
  6. Christopher Chou, Lisa Dunlap, Koki Mashita, Krishna Mandal, Trevor Darrell, Ion Stoica, Joseph E Gonzalez, and Wei-Lin Chiang. 2025. Visionarena: 230k real world user-vlm conversations with preference labels. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3877–3887. find
  7. Martin A. Conway and Christopher W. Pleydell-Pearce. 2000. The construction of autobiographical memories in the self-memory system. Psychological Review, 107(2):261–288. find
  8. Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinyeong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, and Bryan Perozzi. 2024. Test of time: A benchmark for evaluating LLMs on temporal reasoning. arXiv preprint arXiv:2406.09170. find
  9. Sen Hu, Zhiyu Zhang, Yuxiang Wei, Xueran Han, Zhenheng Tang, Huacan Wang, and Ronghao Chen. 2026. CloneMem: Benchmarking long-term memory for AI clones. arXiv preprint arXiv:2601.07023. find
  10. Yuanzhe Hu, Yu Wang, and Julian McAuley. 2025a. Evaluating memory in LLM agents via incremental multi-turn interactions. Preprint, arXiv:2507.05257. — [arXiv:2507.05257](https://arxiv.org/abs/2507.05257) find
  11. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, and 1 others. 2025b. Memory in the age of AI agents. arXiv preprint arXiv:2512.13564. — [arXiv:2512.13564](https://arxiv.org/abs/2512.13564) find
  12. Luoma Ke, Song Tong, Peng Cheng, and Kaiping Peng. 2025. Exploring the frontiers of LLMs in psychological applications: A comprehensive review. Artificial Intelligence Review, 58(10):305. find
  13. Leon OH Kroczek, Alexander May, Selina Het-tenkofer, Andreas Ruider, Bernd Ludwig, and Andreas Mühlberger. 2025. The influence of persona and conversational task on social interactions with a LLM-controlled embodied conversational agent. Computers in Human Behavior, page 108759. find
  14. Zhiyu Li, Shichao Song, Chenyang Xi, Hanyu Wang, Chen Tang, Simin Niu, Ding Chen, Jiawei Yang, Chunyu Li, Qingchen Yu, and 1 others. 2025. MemOS: A memory os for AI system. arXiv preprint arXiv:2507.03724. — [arXiv:2507.03724](https://arxiv.org/abs/2507.03724) find
  15. Xiaochen Liu and Yanan Zhang. 2025. Etimeline: An extensive timeline generation dataset based on large language model. arXiv preprint arXiv:2502.07474. — [arXiv:2502.07474](https://arxiv.org/abs/2502.07474) find
  16. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024a. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851– 138… find
  17. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024b. Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753. find
  18. Dan P. McAdams and Kate C. McLean. 2013. Narrative identity. Current Directions in Psychological Science, 22(3):233–238. find
  19. Duc Cuong Nguyen and Catherine Welch. 2026. Generative artificial intelligence in qualitative data analysis: Analyzing—or just chatting? Organizational Research Methods, 29(1):3–39. find
  20. Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, and Zhenhua Dong. 2025. MemBench: Towards more comprehensive evaluation on the memory of LLM-based agents. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19336–19352, Vienna, Austria. Association for Computational L… find
  21. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025. LongMemEval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth International Conference on Learning Representations (ICLR).
  22. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-Mem: Agentic memory for LLM agents. Preprint, arXiv:2502.12110.
  23. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19724–19731. find
  24. Charles Packer, Vivian Fang, Shishir G Patil, Kevin Lin, Sarah Wooders, and Joseph E Gonzalez. 2023. MemGPT: Towards LLMs as operating systems. Preprint, arXiv:2309.10677. find
  25. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST). find
  26. Muhammad Reza Qorib, Qisheng Hu, and Hwee Tou Ng. 2025. Just what you desire: Constrained timeline summarization with self-reflection for enhanced relevance. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 25065–25073. find
  27. Miao Su, Yucan Guo, Zhongni Hou, Long Bai, Zixuan Li, Yufei Zhang, Guojun Yin, Wei Lin, Xiaolong Jin, Jiafeng Guo, and Xueqi Cheng. 2026. Beyond dialogue time: Temporal semantic memory for personalized LLM agents. arXiv preprint arXiv:2601.07468.
  28. Lipeipei Sun, Tianzi Qin, Anran Hu, Jiale Zhang, Shuo-jia Lin, Jianyan Chen, Mona Ali, and Mirjana Prpa. 2025. Persona-L has entered the chat: Leveraging LLMs and ability-based framework for personas of people with complex needs. In Proceedings of the 2025 CHI Conference on Human Factors in Computin… find
  29. Annalisa Szymanski, Noah Ziems, Heather A Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A Metoyer. 2025. Limitations of the LLM-as-a-judge approach for evaluating LLM outputs in expert knowledge tasks. In Proceedings of the 30th International Conference on Intelligent User Interfaces, pages… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-knowme-bench-benchmarking-person-understanding/