Elamur Research⌕
☰
home / papers / 2026 / Evaluating Memory Capability in Continuous Lifelog Scenario

Evaluating Memory Capability in Continuous Lifelog Scenario

Paper · 2026 · Jianjie Zheng, Zhichen Liu, Zhanyu Shen et al.

Summary

Smart glasses and pocket recorders with always-on microphones can capture a person's everyday spoken interactions. Existing memory benchmarks mostly cover chatbot sessions or two-person dialogues, so they do not test this setting. The authors build LifeDialBench, which has two text subsets of multi-party, timestamped conversation logs:

  • EgoMem is derived from the real egocentric EgoLife videos through bottom-up summarisation.
  • LifeMem is a year-long log of a simulated virtual community. It is generated top-down with LLMs and checked by humans.

Questions are produced from event summaries at several temporal granularities. The authors also introduce an Online Evaluation protocol. Memory is filled in chronological order, and each query can only use information recorded before it, which prevents leakage of future context.

The central result is surprising: a plain RAG baseline over raw transcripts beats more elaborate memory systems. The authors attribute this to summarisation and structuring that discard useful detail. Temporal questions remain hard for every method. The work suggests lifelog assistants should keep context at high fidelity and be evaluated under strict temporal causality.

Key points

  • EgoMem: built from EgoLife, which records six people over seven days. It contains about 1.7k sessions.
  • LifeMem: spans one year with about 3.8k sessions. All dialogues and summaries in the benchmark were generated with Qwen3-235B-A22B-Instruct-2507.
  • Question types: single_event, event_detail, multi_event and time_query. Multi-hop reasoning and temporal grounding are the most difficult.
  • Offline vs. online: for questions answered correctly online but missed offline on LifeMem, offline accuracy reaches only 34.91%. This holds even when retrieval is limited to memories created before the query and top-k is raised from 20 to 100. The authors read this as evidence that offline memory construction overwrites needed information.
  • Systems and resources: the compared systems are RAG, A-Mem, Mem0 and MemOS. Under online evaluation, accuracy drops quarter by quarter, and RAG declines most steeply. Code and data are released at github.com/RayNeo-AI-2025/LifeDialBench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓Shanghai Jiao Tong UniversityShanghai · China212 works · 19,544 citations🎓Southern University of Science and TechnologyShenzhen · China22 works · 1,337 citations🎓Shenzhen UniversityShenzhen · China22 works · 95 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 works

Cite

Cite as Jianjie Zheng, Zhichen Liu, Zhanyu Shen, Jingxiang Qu, Guanhua Chen, Yile Wang, Yang Xu, Yang Liu and Sijie Cheng. 2026. Evaluating Memory Capability in Continuous Lifelog Scenario. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.351

@inproceedings{zheng2026evaluating,
  title = {{Evaluating Memory Capability in Continuous Lifelog Scenario}},
  author = {Jianjie Zheng, Zhichen Liu, Zhanyu Shen, Jingxiang Qu, Guanhua Chen, Yile Wang, Yang Xu, Yang Liu and Sijie Cheng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.351},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.351}
}

References

30 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2024. Longlora: Efficient fine-tuning of long-context large language models. Preprint, arXiv:2309.12307.
  2. Sijie Cheng, Kechen Fang, Yangyang Yu, Sicheng Zhou, Bohao Li, Ye Tian, Tingguang Li, Lei Han, and Yang Liu. 2024. Videgothink: Assessing egocentric video understanding capabilities for embodied ai. arXiv preprint arXiv:2410.11623. — [arXiv:2410.11623](https://arxiv.org/abs/2410.11623) find
  3. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. Preprint, arXiv:2504.19413.
  4. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, and Others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  5. Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2025. Hipporag: neurobiologically inspired long-term memory for large language models. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associa…
  6. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. 2024. RULER: What’s the real context size of your long-context language models? In First Conference on Language Modeling.
  7. Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo J. Taylor, and Dan Roth. 2025. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale. Preprint, arXiv:2504.14225. — [arXiv:2504.14225](https://arxiv.org/abs/2504.14225) find
  8. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledge-intensive nlp tasks. Preprint, arXiv:2005.11401.
  9. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.
  10. Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2024. LooGLE: Can long-context language models understand long contexts? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16304–16333, Bangkok, Thailand. Association for C…
  11. Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, and Zhenhua Dong. 2025. Membench: Towards more comprehensive evaluation on the memory of llm-based agents. Preprint, arXiv:2506.21605. find
  12. Zhiyu Li, Shichao Song, Chenyang Xi, Hanyu Wang, Chen Tang, Simin Niu, Ding Chen, Jiawei Yang, Chunyu Li, Qingchen Yu, and 1 others. 2025. Memos: A memory os for ai system. arXiv preprint arXiv:2507.03724. — [arXiv:2507.03724](https://arxiv.org/abs/2507.03724) find
  13. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  14. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851– 1387… find
  15. Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova, and Radu State. 2025. Limitations of normalization in attention mechanism. Preprint, arXiv:2508.17821. — [arXiv:2508.17821](https://arxiv.org/abs/2508.17821) find
  16. Shulin Tian, Ruiqi Wang, Hongming Guo, Penghao Wu, Yuhao Dong, Xiuying Wang, Jingkang Yang, Hao Zhang, Hongyuan Zhu, and Ziwei Liu. 2025. Ego-r1: Chain-of-tool-thought for ultra-long egocentric video reasoning. arXiv preprint arXiv:2506.13654. — [arXiv:2506.13654](https://arxiv.org/abs/2506.13654) find
  17. Bing Wang, Xinnian Liang, Jian Yang, Hui Huang, Shuangzhi Wu, Peihao Wu, Lu Lu, Zejun Ma, and Zhoujun Li. 2025. Scm: Enhancing large language model with self-controlled memory framework. Preprint, arXiv:2304.13343. — [arXiv:2304.13343](https://arxiv.org/abs/2304.13343) find
  18. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2024. Longmemeval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth International Conference on Learning Representations.
  19. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110.
  20. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, and Others. 2025a. Qwen3 technical report. Preprint, arXiv:2505.09388.
  21. OpenAI. 2022. Chatgpt. Accessed: September 15, 2024. find
  22. Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin, Marco Cominelli, Zhongang Cai, Bo Li, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Widmer, and 3 others. 2025b. Egolife: Towards egoc… find
  23. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and Others. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  24. Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2023. Gpt4tools: Teaching large language model to use tools via self-instruction. Advances in Neural Information Processing Systems, 36:71995–72007.
  25. Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, and Joseph E. Gonzalez. 2023. Memgpt: Towards llms as operating systems. CoRR, abs/2310.08560. find
  26. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. Textgrad: Automatic "differentiation" via text. Preprint, arXiv:2406.07496.
  27. Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: a temporal knowledge graph architecture for agent memory. arXiv preprint arXiv:2501.13956.
  28. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. 2023. find
  29. Judging NAIH. 2025. GitHub -gkamradt/LLMTest_NeedleInAHaystack: Doing simple retrieval from LLM models at various context lengths to measure accuracy — github.com. https://github.com/gkamradt/LLMTest_ NeedleInAHaystack. [Accessed 18-09-2025]. llm-as-a-judge with mt-bench and chatbot arena. In Advanc… ↗
  30. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fo… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-evaluating-memory-capability-continuous-lifelog/