Elamur Research⌕
☰
home / papers / 2026 / Hybrid Self-evolving Structured Memory for Computer-Use Agents

Hybrid Self-evolving Structured Memory for Computer-Use Agents

Paper · 2026 · Sibo Zhu, Wenyi WU, Kun Zhou et al.

Summary

Agents that operate computers through vision–language models (VLMs) often fail on long, multi-step tasks. They make mistakes partway through or miss important constraints. Earlier memory add-ons store past trajectories as flat lists, either as text summaries or as dense vectors, and retrieve them by similarity alone. The authors propose HyMEM, a memory organized as a graph and loosely modeled on how hippocampus and neocortex divide work. Each node holds three things for one successful trajectory: a short strategy heuristic, semantic attribute tags, and compact CoMEM embeddings of the full episode. Nodes are linked when they share tags, so retrieval can hop across related experiences. As new trajectories arrive, a VLM judge decides whether to add, merge or replace nodes, which keeps the memory from growing unchecked. During execution, the agent's working memory is re-retrieved whenever the task enters a new phase. With HyMEM, 7B/8B open models match or beat closed systems like GPT-4o and Gemini2.5-Pro-Vision at a small fraction of the cost.

Key points

  • Qwen2.5-VL-7B rises from 12.5% to 35.0% (+22.5 points), beating Gemini2.5-Pro-Vision by 5.4% and GPT-4o by 15.3%. Similar gains are reported for UI-TARS-1.5-7B and Qwen3-VL-8B.
  • Memory construction works in stages:
    • CLIP text and image vectors feed a FAISS nearest-neighbor search.
    • A strict 0.92 cosine threshold narrows the candidates.
    • At most one lightweight VLM judge call is made per new trajectory.
  • On Amazon, HyMEM with Qwen2.5-VL-7B reaches 63.4% at about $0.01 per task. This equals Claude-4 and far exceeds GPT-4o (24.4%, $0.38). Cost is overall 20–30× lower than closed models.
  • In the ablation on Coursera, the hybrid memory alone (no graph expansion or self-evolution) lifts accuracy from 2.4% to 47.6%. The full system reaches 54.8%.
  • Limitations: the update decisions are heuristic rather than learned, for example via RL. Backbones larger than 8B (32B/70B) were not tested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California San DiegoSan Diego · United States89 works · 54,032 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

Cite

Cite as Sibo Zhu, Wenyi WU, Kun Zhou, Stephen Wang and Biwei Huang. 2026. Hybrid Self-evolving Structured Memory for Computer-Use Agents. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.549

@inproceedings{zhu2026hybrid,
  title = {{Hybrid Self-evolving Structured Memory for Computer-Use Agents}},
  author = {Sibo Zhu, Wenyi WU, Kun Zhou, Stephen Wang and Biwei Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.549},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.549}
}

References

36 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Claude 4. https://www.anthropic.com/index/claude-4. Accessed: 2025-09-21. ↗
  2. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. Qwen3-vl technical report. Preprint, a…
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025b. Qwen2.5-vl technical report. Preprint, arXiv:2502.1…
  4. Tanvir Bhathal and Asanshay Gupta. 2025. Websight: A vision-first architecture for robust web agents. arXiv preprint arXiv:2508.16987. — [arXiv:2508.16987](https://arxiv.org/abs/2508.16987) find
  5. Liangyu Chen, Hanzhang Zhou, Chenglin Cai, Jianan Zhang, Panrong Tong, Quyu Kong, Xu Zhang, Chen Liu, Yuqi Liu, Wenxuan Wang, Yue Wang, Qin Jin, and Steven Hoi. 2025. Ui-ins: Enhancing gui grounding with multi-perspective instruction-as-reasoning. Preprint, arXiv:2510.20286. — [arXiv:2510.20286](https://arxiv.org/abs/2510.20286) find
  6. Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. Guicourse: From general vision language models to versatile gui agents. find
  7. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2web: Towards a generalist agent for the web. In Advances in Neural Information Processing Systems, volume 36, pages 28091–28114. Curran Associates, Inc.
  8. Gemini Team Google. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  9. Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. 2024. Webvoyager: Building an end-to-end web agent with large multimodal models. arXiv preprint arXiv:2401.13919.
  10. Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. Cogagent: A visual language model for gui agents. Preprint, arXiv:2312.08914.
  11. Mengkang Hu, Tianxing Chen, Qiguang Chen, Yao Mu, Wenqi Shao, and Ping Luo. 2025a. HiAgent: Hierarchical working memory management for solving long-horizon agent tasks with large language model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon…
  12. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 others. 2025b. Memory in the age of ai agents. Preprint, arX… — [arXiv:2512.13564](https://arxiv.org/abs/2512.13564) find
  13. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017. Billion-scale similarity search with gpus. Preprint, arXiv:1702.08734.
  14. Tiange Luo, Lajanugen Logeswaran, Justin Johnson, and Honglak Lee. 2025. Visual test-time scaling for gui agent grounding. Preprint, arXiv:2505.00684. — [arXiv:2505.00684](https://arxiv.org/abs/2505.00684) find
  15. James L. McClelland, Bruce L. McNaughton, and Randall C. O’Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3):419–457. find
  16. OpenAI. 2024. Gpt-4o technical report. https://openai.com/index/hello-gpt-4o/. Accessed September 2025. ↗
  17. Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2025. Reasoningbank: Scaling agent self–evolving with reasoning memory.… — [arXiv:2509.25140](https://arxiv.org/abs/2509.25140) find
  18. Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2024. Memgpt: Towards llms as operating systems. Preprint, arXiv:2310.08560. find
  19. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, and 16 others. 2025. Ui-tars: Pioneering automated gui interaction with…
  20. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. Preprint, arXiv:2103.00020.
  21. Zhili Shen, Chenxin Diao, Pavlos Vougiouklis, Pascual Merita, Shriram Piramanayagam, Enting Chen, Damien Graux, Andre Melo, Ruofei Lai, Zeren Jiang, Zhongyang Li, Ye Qi, Yang Ren, Dandan Tu, and Jeff Z. Pan. 2025. GeAR: Graph-enhanced agent for retrieval-augmented generation. In Findings of the Asso… find
  22. Selena Song, Ziming Xu, Zijun Zhang, Kun Zhou, Jiaxian Guo, Lianhui Qin, and Biwei Huang. 2025. Learning plug-and-play memory for guiding video diffusion models. Preprint, arXiv:2511.19229. — [arXiv:2511.19229](https://arxiv.org/abs/2511.19229) find
  23. Eleanor Spens and Neil Burgess. 2024a. Consolidation of sequential experience into a deep generative network explains human memory, prediction and planning. bioRxiv. Preprint. find
  24. Eleanor Spens and Neil Burgess. 2024b. A generative model of memory construction and consolidation. Nature Human Behaviour, 8(3):526–543. find
  25. Shulin Tian, Ziniu Zhang, Liangyu Chen, and Ziwei Liu. 2025. Mmina: Benchmarking multihop multimodal internet agents. Preprint, arXiv:2404.09992. find
  26. Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2024. Agent workflow memory: Continually inducing and applying workflows for web agents. Preprint, arXiv:2409.07429.
  27. Samantha E. Williams, Jaclyn H. Ford, and Elizabeth A. Kensinger. 2022. The power of negative and positive episodic memories. Cognitive, Affective, & Behavioral Neuroscience, 22(5):869–903. find
  28. Wenyi Wu, Zixuan Song, Kun Zhou, Yifei Shao, Zhiting Hu, and Biwei Huang. 2025a. Towards general continuous memory for vision-language models. Preprint, arXiv:2505.17670. — [arXiv:2505.17670](https://arxiv.org/abs/2505.17670) find
  29. Wenyi Wu, Kun Zhou, Ruoxin Yuan, Vivian Yu, Stephen Wang, Zhiting Hu, and Biwei Huang. 2025b. Auto-scaling continuous memory for gui agent. Preprint, arXiv:2510.09038. ArXiv preprint arXiv:2510.09038. — [arXiv:2510.09038](https://arxiv.org/abs/2510.09038) find
  30. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. Amem: Agentic memory for LLM agents. Preprint, arXiv:2502.12110.
  31. Leyang Yang, Ziwei Wang, Xiaoxuan Tang, Sheng Zhou, Dajun Chen, Wei Jiang, and Yong Li. 2025. Probench: Benchmarking gui agents with accurate process information. Preprint, arXiv:2511.09157. — [arXiv:2511.09157](https://arxiv.org/abs/2511.09157) find
  32. Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. Appagent: Multimodal agents as smartphone users. Preprint, arXiv:2312.13771.
  33. Guibin Zhang, Muxin Fu, Guancheng Wan, Miao Yu, Kun Wang, and Shuicheng Yan. 2025a. G-memory: Tracing hierarchical memory for multi-agent systems. Preprint, arXiv:2506.07398. — [arXiv:2506.07398](https://arxiv.org/abs/2506.07398) find
  34. Guibin Zhang, Muxin Fu, and Shuicheng Yan. 2025b. Memgen: Weaving generative latent memory for self-evolving agents. Preprint, arXiv:2509.24704. — [arXiv:2509.24704](https://arxiv.org/abs/2509.24704) find
  35. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Expel: Llm agents are experiential learners. Preprint, arXiv:2308.10144. find
  36. Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024. Gpt-4v(ision) is a generalist web agent, if grounded.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2026-hybrid-self-evolving-structured-memory/