Elamur Research⌕
☰
home / papers / 2026 / What Deserves Memory: Adaptive Memory Distillation for LLM Agents

What Deserves Memory: Adaptive Memory Distillation for LLM Agents

Paper · 2026 · Wenquan Ma, Jiayan Nan and WenLong Wu

Summary

LLM agents in long interactions need memory, but deciding which experiences to keep is usually handled by hand-crafted rules such as importance ratings, emotion labels or fact templates. These rules reflect designer assumptions and can either distort information or inflate storage. The authors propose NEMORI, a training-free framework built on an idea from Predictive Coding Theory. Content the agent could already anticipate from its stored knowledge is treated as redundant, and prediction errors signal what is worth keeping.

The pipeline has two cascading modules. Episodic Memory Integration uses an LLM to split buffered messages into coherent episodes and rewrite them as narratives indexed by embeddings. Semantic Knowledge Distillation then extracts what those episodes reveal beyond expectations. NEMORI addresses only the distillation step, so it can feed other memory managers.

On LoCoMo and LongMemEval S, NEMORI matches or exceeds full-context prompting while using far fewer tokens, and its advantage grows with context length. The work offers a data-driven alternative to heuristic memory design.

Key points

  • On LongMemEval S (~105K tokens per history), NEMORI beats Full Context by +16.7% with gpt-4o-mini (average 55.0 → 64.2) and +13.7% with gpt-4.1-mini (65.6 → 74.6), using 95–96% less context. On the shorter LoCoMo the gains are marginal (+1.0% and +0.2%).
  • Used as a distillation layer for the third-party systems A-MEM and MemoryOS, it cuts stored memory by 45–64%. Average scores stay within about ±4%, and core scores (excluding Temporal) improve by 1.9–6.1%.
  • Ablations show both memory types are needed. With gpt-4o-mini, dropping semantic retrieval lowers the score from 73.0 to 54.7, and dropping episodic retrieval lowers it to 65.0.
  • Indexing by narrative-episode embeddings outperforms raw-episode embeddings (e.g., 77.0 vs. 75.3 when raw episodes are retrieved). Performance levels off at Top-K ≈ 10, above Full Context.
  • Code is released at github.com/nemori-ai/nemori. The authors note that management and retrieval are deliberately simple, and that integration interfaces are still conceptual.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓Beihang UniversityBeijing · China65 works · 8,452 citations🎓Shanghai University of Finance and EconomicsShanghai · China13 works · 825 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksNeuroscience · 165 worksCognitive Neuroscience · 164 works

🧭 Topics 1Research subjects and languages the work is about.Efficient models & compression · 242 works

Cite

Cite as Wenquan Ma, Jiayan Nan and WenLong Wu. 2026. What Deserves Memory: Adaptive Memory Distillation for LLM Agents. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1607

@inproceedings{ma2026what,
  title = {{What Deserves Memory: Adaptive Memory Distillation for LLM Agents}},
  author = {Wenquan Ma, Jiayan Nan and WenLong Wu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1607},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1607}
}

References

24 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Petr Anokhin, Nikita Semenov, Artyom Y. Sorokin, Dmitry Evseev, Andrey Kravchenko, Mikhail Burtsev, and Evgeny Burnaev. 2025. Arigraph: Learning knowledge graph world models with episodic memory for LLM agents. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intellig… find
  2. Harrison Chase. 2022. Langchain. https://github.com/langchain-ai/langchain. Accessed: 2025-12-31. ↗
  3. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. CoRR, abs/2504.19413.
  4. ICKG 2023, Shanghai, China, December 1-2, 2023, pages 120–127. IEEE. find
  5. Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. 2025. Memory OS of AI agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25972–25981, Suzhou, China. Association for Computational Linguistics.
  6. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Proc…
  7. Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. 2025a. Hello again! LLM-powered personalized agent for long-term dialogue. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo…
  8. Rui Li, Zeyu Zhang, Xiaohe Bo, Zihang Tian, Xu Chen, Quanyu Dai, Zhenhua Dong, and Ruiming Tang. 2025b. CAM: A constructivist view of agentic memory for LLM-based reading comprehension. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  9. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  10. Andy Clark. 2013. Whatever next? predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences, 36(3):181–204. find
  11. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851– 1387… find
  12. Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. 2025. Lightmem: Lightweight and efficient memory-augmented generation. CoRR, abs/2510.18866. — [arXiv:2510.18866](https://arxiv.org/abs/2510.18866) find
  13. James L McClelland, Bruce L McNaughton, and Randall C O’Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419. find
  14. Karl Friston. 2010. The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2):127–138. find
  15. Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, and Joseph E. Gonzalez. 2023. Memgpt: Towards llms as operating systems. CoRR, abs/2310.08560. find
  16. Le Huang, Hengzhi Lan, Zijun Sun, Chuan Shi, and Ting Bai. 2024. Emotional RAG: enhancing role-playing agents through emotional retrieval. In IEEE International Conference on Knowledge Graph, Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H.… find
  17. Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA,… find
  18. Rajesh PN Rao and Dana H Ballard. 1999. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79–87. find
  19. Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: A temporal knowledge graph architecture for agent memory. CoRR, abs/2501.13956. find
  20. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025. Longmemeval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net.
  21. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-mem: Agentic memory for LLM agents. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
  22. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fo… find
  23. Knowledge Consolidation. New insights are directly merged into the summary: S sum ← f LLM (P sum ∥ K in ∥ S sum ) where P sum instructs the LLM to merge distilled insights into the summary. A.2 Variant: Naive RAG This configuration serves as an ablation in Section 4.4, applying no management to sema… find
  24. This appendix details three instantiations of the management interfaces defined in Section 3.3, demonstrating N EMORI ’s architectural flexibility. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ma-2026-what-deserves-memory-adaptive-memory/