Elamur Research⌕
☰
home / papers / 2026 / Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory

Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory

Paper · 2026 · Zihao Tang, Xin Yu, Ziyu Xiao et al.

Summary

Long-running LLM agents need a memory of past conversations. Current RAG and Graph-RAG systems retrieve that memory mainly by similarity, through embeddings or BM25. This fast, "System-1" style of lookup misses evidence that is only weakly related to the query on the surface. It also struggles when a question needs a complete, global view of the history, such as listing every city a person visited. Mnemis, from Microsoft and built on Graphiti, adds a second, deliberate "System-2" route called Global Selection. Memory lives in two structures:

  • Base graph: stores Episodes, Entities, Edges and Episodic Edges, with improved extraction, reflection and de-duplication, and supports similarity search.
  • Hierarchical graph: an LLM groups entities bottom-up into multi-level Categories. At query time the LLM browses these categories from the top layer down to select relevant entities and everything linked to them.

Results from both routes are merged and re-ranked. With GPT-4.1-mini, Mnemis reaches state-of-the-art scores on LoCoMo and LongMemEval-S. This shows that structured, top-down browsing complements similarity retrieval for long-term memory.

Key points

  • Main results: with GPT-4.1-mini, Mnemis scores 93.9 on LoCoMo and 91.6 on LongMemEval-S. Code is released at github.com/microsoft/Mnemis.
  • Hierarchy construction rules: the hierarchy follows three principles: Minimum Concept Abstraction, Many-to-Many Mapping (a node may have several parent Categories), and a Compression Efficiency Constraint (each Category needs a minimum number of children, and higher layers cannot have more Categories than lower ones).
  • Ablation on LoCoMo (overall LLM-as-a-Judge score):
    • System-1 Re-ranked: 89.1
    • System-2 Only: 87.7
    • Both routes combined: 93.3, with the largest gains on Multi-Hop (91.8) and Temporal (90.3) questions.
  • Re-rankers: the main experiments use Qwen3-Reranker-8B. Smaller re-rankers cost little accuracy: Qwen3-Reranker-0.6B scores 92.6 and BGE-Reranker-V2-M3 scores 92.7, against 93.3 for the 8B model.
  • Difference from GraphRAG: GraphRAG builds single-parent communities with community detection and answers per community. Mnemis allows many-to-many category membership and produces one answer from the aggregated retrieved context.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksInformation retrieval & search · 473 works

Cite

Cite as Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun and Qi Zhang. 2026. Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1096

@inproceedings{tang2026mnemis,
  title = {{Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory}},
  author = {Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun and Qi Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1096},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1096}
}

References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Arslan, Hussam Ghanem, Saba Munawar, and Christophe Cruz. 2024. A survey on rag with llms. Procedia computer science, 246:3781–3790. find
  2. Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. 2024. Titans: Learning to memorize at test time. arXiv preprint arXiv:2501.00663. find
  3. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216.
  4. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. Preprint, arXiv:2504.19413.
  5. Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758–759.
  6. Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130. find
  7. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. Preprint, arXiv:2503.09516.
  8. Daniel Kahneman. 2011. macmillan. find
  9. Sangyeop Kim, Yohan Lee, Sanghwa Kim, Hyunjong Kim, and Sungzoon Cho. 2025. Pre-storage reasoning for episodic memory: Shifting inference burden to memory for personalized dialogue. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 22096–22113. find
  10. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–…
  11. Zhiyu Li, Chenyang Xi, Chunyu Li, Ding Chen, Boyu Chen, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Chen Tang, Qingchen Yu, Jihao Zhao, Yezhaohui Wang, Peng Liu, Zehao Lin, Pengyuan Wang, Jiahao Huo, Tianyi Chen, Kai Chen, and 20 others. 2025a. Memos: A memory os for ai system. Preprint, arXiv… — [arXiv:2507.03724](https://arxiv.org/abs/2507.03724) find
  12. Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, and Cheng-Lin Liu. 2025b. Tl;dr: Too long, do reweighting for efficient llm reasoning compression. Preprint, arXiv:2506.02678.
  13. Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Retrieval augmented generation or long-context llms? a comprehensive study and hybrid approach. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 881– 893. find
  14. Xiao Liang, Zhongzhi Li, Yeyun Gong, Yelong Shen, Ying Nian Wu, Zhijiang Guo, and Weizhu Chen. 2025. Beyond pass@1: Self-play with variational problem synthesis sustains rlvr. Preprint, arXiv:2508.14029. — [arXiv:2508.14029](https://arxiv.org/abs/2508.14029) find
  15. Zhenghao Lin, Zihao Tang, Xiao Liu, Yeyun Gong, Yi Cheng, Qi Chen, Hang Li, Ying Xin, Ziyue Yang, Kailai Yang, Yu Yan, Xiao Liang, Shuai Lu, Yiming Huang, Zheheng Luo, Lei Qu, Xuan Feng, Yaoxiang Wang, Yuqing Xia, and 15 others. 2025. Sigma: Differential rescaling of query, key and value for efficie… — [arXiv:2501.13629](https://arxiv.org/abs/2501.13629) find
  16. Jiaheng Liu, Dawei Zhu, Zhiqi Bai, Yancheng He, Huanxuan Liao, Haoran Que, Zekun Wang, Chenchen Zhang, Ge Zhang, Jiebin Zhang, Yuanxing Zhang, Zhuo Chen, Hangyu Guo, Shilong Li, Ziqiang Liu, Yong Shan, Yifan Song, Jiayi Tian, Wenhao Wu, and 18 others. 2025. A comprehensive survey on long context lan…
  17. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753. find
  18. Jiayan Nan, Wenquan Ma, Wenlong Wu, and Yize Chen. 2025. Nemori: Self-organizing agent memory inspired by cognitive science. Preprint, arXiv:2508.03341. — [arXiv:2508.03341](https://arxiv.org/abs/2508.03341) find
  19. Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, and 1 others. 2025. Reasoningbank: Scaling agent self-evolving with reasoning memory. arXiv preprint arXiv:2509.25140. — [arXiv:2509.25140](https://arxiv.org/abs/2509.25140) find
  20. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Jianfeng Gao. 2025. Secom: On memory construction and retrieval for personalized conversational agents. In The Thirteenth International Conference on Learning Represe… find
  21. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2024. YaRN: Efficient context window extension of large language models. In The Twelfth International Conference on Learning Representations. find
  22. Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: A temporal knowledge graph architecture for agent memory. Preprint, arXiv:2501.13956. find
  23. Henrique* Schechter Vera, Sahil* Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, Daniel Cer, Alice Lisak, Min Choi, Lucas Gonzalez, Omar Sanseviero, Glenn Cameron, Ian Ballantyne, Kat Black, Kaifeng Chen, and 69 others. 2025. Embe… find
  24. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.
  25. Zihao Tang, Zheqi Lv, Shengyu Zhang, Fei Wu, and Kun Kuang. 2024. Modelgpt: Unleashing llm’s capabilities for tailored model generation. Preprint, arXiv:2402.12408. — [arXiv:2402.12408](https://arxiv.org/abs/2402.12408) find
  26. Endel Tulving and 1 others. 1972. Episodic and semantic memory. Organization of memory, 1(381-403):1. find
  27. Liang Wang, Haonan Chen, Nan Yang, Xiaolong Huang, Zhicheng Dou, and Furu Wei. 2025. Chain-of-retrieval augmented generation. In NeurIPS 2025. find
  28. Yu Wang and Xi Chen. 2025. Mirix: Multi-agent memory system for llm-based agents. Preprint, arXiv:2507.07957. — [arXiv:2507.07957](https://arxiv.org/abs/2507.07957) find
  29. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2024a. Longmemeval: Benchmarking chat assistants on long-term interactive memory.
  30. Tao Wu, Jingyuan Chen, Wang Lin, Mengze Li, Yumeng Zhu, Ang Li, Kun Kuang, and Fei Wu. 2025a. Embracing imperfection: Simulating students with diverse cognitive levels using llm-based agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P…
  31. Tao Wu, Jingyuan Chen, Wang Lin, Jian Zhan, Mengze Li, Kun Kuang, and Fei Wu. 2025b. Personalized distractor generation via mcts-guided reasoning reconstruction. CoRR, abs/2508.11184. — [arXiv:2508.11184](https://arxiv.org/abs/2508.11184) find
  32. Tao Wu, Mengze Li, Jingyuan Chen, Wei Ji, Wang Lin, Jinyang Gao, Kun Kuang, Zhou Zhao, and Fei Wu. 2024b. Semantic alignment for multimodal large language models. In Proceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 - 1 November 2… find
  33. Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Kristian Kersting, Jeff Z Pan, Hinrich Schütze, and 1 others. 2025. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint arXiv:2508.19828.
  34. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  35. Sizhe Zhou. 2025. A simple yet strong baseline for long-term conversational memory of llm agents. Preprint, arXiv:2511.17208. — [arXiv:2511.17208](https://arxiv.org/abs/2511.17208) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tang-2026-mnemis-dual-route-retrieval-hierarchical/